SharePoint - Using the REST Interface & Using the Client Object Model

Using the REST Interface

The SharePoint 2010 Representational State Transfer (REST) interface is a WCF Data Service that allows you to use construct HTTP requests to query SharePoint list data. Like all RESTful Web services, the SharePoint REST interface maps HTTP verbs to data operations, as shown in the following table.

HTTP verb	Data operation
GET	Retrieve
POST	Create
PUT	Update (update all fields and use default values for any undefined fields)
DELETE	Delete
MERGE	Update (update only the fields that are specified and changed from current version)

Note:

In practice, many firewalls and other network intermediaries block HTTP verbs other than GET and POST. To work around this issue, WCF Data Services (and the OData standard) support a technique known as "verb tunneling." In this technique, PUT, DELETE, and MERGE requests are submitted as a POST request, and an X-HTTP-Method header specifies the actual verb that the recipient should apply to the request. For more information, see X-HTTP-Method on MSDN and OData: Operations (the Method Tunneling through POST section) on the OData Web site.

A RESTful service models data entities, in this case SharePoint lists, as HTTP resources that can be addressed by a URL. You can append query strings to the URLs in order to specify filter criteria or query logic. The following examples show some URLs that correspond to simple REST operations.

http://localhost/_vti_bin/listdata.svc/Parts

The preceding URL returns the contents of the Parts list in XML format as an Atom feed.

http://localhost/_vti_bin/listdata.svc/Parts(3)

The preceding URL returns the Parts list item with an ID value of 3 as an Atom feed.

http://localhost/_vti_bin/listdata.svc/Parts?$orderby=Name

The preceding URL returns the Parts list as an Atom feed, ordered by the Name field.

However, you don't need to manually construct HTTP requests in order to use the SharePoint REST interface. When you use Visual Studio 2010 to create a SharePoint client application, Visual Studio will generate a WCF Data Services Framework service proxy when you add a reference to the service. The service proxy provides strongly-typed entity classes and enables you to use LINQ expressions to build queries. Behind the scenes, the service proxy manages the details of building and submitting requests to the service.

The SharePoint REST interface is based on the REST-based Open Data protocol (OData) for Web-based data services, which extends the Atom and AtomPub syndication formats to exchange XML data over HTTP. Because OData is a platform-independent open standard, the SharePoint REST interface is a great way to access SharePoint list data from platforms on which the CSOM may be unavailable to you, such as from non-Windows–based operating systems. However, the REST interface only provides access to list data—if you need to manipulate other data on your SharePoint site, you will need to use the CSOM. The REST implementation can also return the output in JavaScript Object Notation (JSON) format as an alternative to an ATOM feed. JSON is a compact representation of the returned results that can be easily parsed by JavaScript clients.

Note:

For background information about Windows Communication Foundation (WCF)–based REST services, see Overview of REST in WCF on MSDN. For product documentation for the SharePoint REST interface, see SharePoint Foundation REST Interface on MSDN. For more information about creating a service proxy for the REST interface, see Query SharePoint Foundation with ADO.NET Data Services on MSDN. For more information about OData, Atom, and REST, see Open Data Protocol by Example on MSDN and the Open Data Protocol Web site. ADO.NET Data Services and WCF Data Services are the same thing—WCF Data Services is now the official name.

Using the Service Proxy

The WCF Data Services service proxy for the SharePoint REST interface includes classes that provide strongly-typed representations of the lists and content types in your SharePoint site. At the top level in the service proxy object model, a data context class that inherits from DataServiceContext provides the starting point for all your service calls. In this way, it performs a similar role to the data context class that you use with the LINQ to SharePoint provider. When you want to perform a data operation using the REST interface, your first step is always to instantiate the data context class to the WCF service endpoint for your SharePoint site. For example, in the Client Reference Implementation, the data context class is named PartsDataContext.

C#

PartsDataContext context = new PartsDataContext(
  new Uri("http://localhost/sites/sharepointlist/_vti_bin/listdata.svc"));

The data context class allows you to specify contextual information for the operations, such as user credentials, and it provides methods that you can use to build and execute REST queries. Your code runs in a browser—using either JavaScript or Silverlight—the calls to SharePoint will use the security context already established by the browser session. You can use the LINQ query syntax to build query expressions. For example, the following method, which was taken from the Client Reference Implementation, retrieves parts that start with the SKU search string.

C#

var partsQuery = (DataServiceQuery)
      context.Parts.Where(p => p.SKU.StartsWith(SearchSku))
        .Select(p => new PartsItem { Title = p.Title, 
                                     SKU = p.SKU, 
                                     Id = p.Id, 
                                     Description = p.Description 
                                   });

// Execute query.
query.BeginExecute(DisplayParts, query);

The preceding example demonstrates various key points of submitting queries to the REST interface by using the service proxy:

• It creates a LINQ expression from the data context object.
• The LINQ expression returns a query in the form of a DataServiceQuery instance. This is a WCF Data Services class that represents a query request to a service, where the query returns a collection of TElementinstances. The DataServiceQuery class implements the IQueryable interface, which allows you to use the LINQ extension methods provided by the Queryable class to query the result set. In the preceding example, TElement is PartsItem. The DataServiceQuery.BeginExecute method is used to send the query to the REST service asynchronously, passing in the callback delegate and the query object as arguments. The results of the query in partsQuery are not available until the callback delegate, DisplayParts, is invoked.
• The proxy generates a REST statement from the LINQ expression and submits it to the REST service, as shown here.
  http://contoso/sites/sharepointlist/_vti_bin/listdata.svc/Parts()
    ?$filter=startswith(SKU,'sku')
      &$select=Title,SKU,Id,Description
  
  

Note:

You can also use the CreateQuery method to create the query object explicitly from the data context. In the previous example, where a LINQ statement was specified without calling CreateQuery, the WCF Data Services proxy created the query object implicitly when the results of the LINQ statement were cast to a DataServiceQueryinstance. For example, the following statement is functionally equivalent to the previous code example: context.CreateQuery("Parts").Where(p => p.SKU.StartsWith(SearchSku)).Select(p => new PartsItem { Title = p.Title, SKU = p.SKU, Id = p.Id, Description = p.Description }); In general, the implicit approach is preferred for readability and simplicity.

The callback method iterates through the query results and adds each item to the Parts observable collection. The Parts collection is bound to a UI control that renders the results in a grid. Because this is a Silverlight application, you must use the Dispatcher.BeginInvoke method to execute the data binding logic asynchronously on the UI thread.

C#

Dispatcher.BeginInvoke(
  () =>
  {
    Parts.Clear(); 
    DataServiceQuery query = 
      (DataServiceQuery) result.AsyncState;
    
    var partResults = query.EndExecute(result);

    foreach (var part in partResults)
    {
      Parts.Add(part);
    }
  });

Although the queries themselves will vary, the pattern of use remains broadly the same for all REST interface queries.

How Does the REST Interface Work?

When you create a WCF Data Services service proxy for the REST interface, you can use it to send a query to the REST interface in one of three ways:

• You can use the data context class to create a DataServiceQuery instance, as seen in the preceding code examples. When you use this approach, you submit a LINQ expression to the service proxy. The service proxy converts the LINQ expression into a URL-based REST request and submits it to the REST interface.
• You can use view projection, in which case the DataServiceQuery is created implicitly. This approach is described in more detail in the next section.
• You can use the data context class to submit a URL-based REST request directly, as shown by the following code example.
  C#

  context.Execute(new
    uri("http://contoso/_vti_bin/listdata.svc/Parts()
         ?$filter=startswith(SKU,'Sku2')
         &$select=Title,SKU,Id,Description"));
  
  

The LINQ expression used on the client is specific to the WCF Data Services proxy, which converts the LINQ expression into a REST statement. On the SharePoint server, the REST service implementation translates the REST statement into a LINQ to SharePoint expression. This translation process is not visible to the developer. The important thing to note is that the LINQ expressions you submit to the service proxy on the client are completely independent of the LINQ to SharePoint expressions that the REST service implementation generates in order to fulfill the request. The LINQ to SharePoint provider converts the server-generated LINQ expressions into CAML, and then it executes the CAML queries against your SharePoint lists. The REST interface returns the results to the service proxy in JSON format or as an Atom feed, using the OData protocol. The service proxy then converts the response into strongly-typed entity instances and returns the results to the caller. The following illustration shows this process.

The SharePoint REST interface

It can be instructive to see how LINQ expressions are translated into REST queries, and how these REST queries translate into the HTTP requests and responses that are exchanged between the client and the server. Using the preceding LINQ expression as an example, the process is as follows:

1. The proxy forms a REST query from the LINQ expression.
   http://localhost/sites/sharepointlist/_vti_bin/listdata.svc/Parts()
     ?$filter=startswith(SKU,'sku')
       &$select=Title,SKU,Id,Description
   
   

2. The proxy submits the REST query to the server as an HTTP GET request.
   HTTP Request
   GET http://localhost/sites/sharepointlist/_vti_bin/listdata.svc/Parts()?$filter=startswith(SKU,'sku1')&$select=Title,SKU,Id,Description HTTP/1.1
   Accept: application/atom+xml,application/xml
   Accept-Language: en-US
   Referer: file:///C:/spg3/Trunk/Source/Client/Client.REST/Client.REST.Silverlight/Bin/Debug/Client.REST.Silverlight.xap
   Accept-Encoding: identity
   DataServiceVersion: 2.0;NetFx
   MaxDataServiceVersion: 2.0;NetFx
   User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.3; MS-RTC LM 8; .NET4.0C; .NET4.0E)
   Host: contoso
   Connection: Keep-Alive
   
   

3. The server responds with an OData-formatted result set. (The HTTP headers have been omitted for brevity.)
   HTTP Response

   
     
     http://contoso/sites/sharepointlist/_vti_bin/listdata.svc/Parts
     2010-05-30T14:20:47Z
     
            
       http://contoso/sites/sharepointlist/_vti_bin/listdata.svc/Parts(2)
       
       
       2010-05-21T14:06:12-04:00
       
         
       
       
       
       
         
           SHAFT - PUMP 1
           SKU1
           
           2
         
       
     
   
   
   

4. The WCF Data Services proxy invokes the DisplayParts delegate and provides the results from the request as a strongly-typed collection of PartsItem instances.

Query Efficiency with the REST Interface

In order to understand the performance implications of using the REST interface, you need to understand what happens when you request data from the service:

1. You submit a LINQ expression to the service proxy.
2. The service proxy converts your LINQ expression into a URL-based REST request, and then submits the request to the REST interface on the server.
3. The REST interface converts the REST request into a LINQ to SharePoint expression.
4. The LINQ to SharePoint provider converts the LINQ expression into a CAML query.

Joins

You have some control over whether the REST interface generates LINQ expressions with efficient syntax. You can gain an insight into the performance of REST-based queries by understanding how some specific REST constructs are implemented by the service.

The REST interface does not support explicit list joins. You can use the Expand method to navigate from one entity to a related entity. Although this appears similar to a join, it actually results in the execution of additional list queries on the server. If required, the REST provider performs an implicit join to satisfy the where clause. However, for each item in the result set, the Expand method causes an additional list query to retrieve the related entity instance that corresponds to the value in a lookup column. For example, consider the following query that retrieves a list of inventory locations ordered by Part.SKU.

C#

var query = (DataServiceQuery)
            context.InventoryLocations
                   .Expand("Part")
                   .Where(p => p.Part.SKU.StartsWith(SearchSku))
                   .OrderBy(p => p.Part.SKU);

The Expand method in our LINQ query is translated to an &expand="Part" query string in the REST request URL, as shown here.

http://contoso/sites/sharepointlist/_vti_bin/listdata.svc/InventoryLocations()
  ?$filter=startswith(Part/SKU,'sku')
    &$orderby=Part/SKU
      &$expand=Part

In this case, the following actions take place in order to execute the query on the server:

1. A list join is performed between the Inventory Locations list and the Parts list, in order to satisfy the where clause match on the part SKU. From the REST statement, the implicit join occurs because Part/SKU in the startswithstatement follows a lookup field relationship.
2. The inventory location items are ordered by part SKU.
3. For each inventory item in the result set, a query is executed on the server to retrieve the related part to satisfy the expand clause.
4. The results are formatted using the OData protocol and returned to the caller.

As you can see, this operation is going to be less efficient than submitting a CAML query with a join predicate where all values can be retrieved in a single list query. However, the CSOM is the only data access mechanism that supports explicit joins. The CSOM allows you to submit a CAML query that specifies a list join and a view projection directly from your client-side logic.

Projections

The REST interface supports view projections. As described in Data Access in SharePoint 2010, view projections improve query efficiency by limiting the field values returned to only those fields that are of interest. For example, the following query uses a view projection to select parts, based on a partial part SKU match.

C#

context.Parts.Where(p => p.SKU.StartsWith(SearchSku))
             .Select(p => new PartsItem {  Title = p.Title, 
                                           SKU = p.SKU, 
                                           Id = p.Id, 
                                           Description = p.Description 
                                        });

The service proxy translates this query into the following REST request URL, and then it parses the response feed into a collection of PartsItem instances.

http://contoso/_vti_bin/listdata.svc/Parts()   
  ?$filter=startswith(SKU,'SKU2') 
    &$select=Title,SKU,Id,Description

You can also perform query projections explicitly on the query object. This can be useful as a concise way to query multiple related entities.

C#

var query = (DataServiceQuery)context.
    CreateQuery("InventoryLocations")
       .Expand("Part")
       .AddQueryOption("$select", 
                       "BinNumber,Quantity,Title,Id,PartId,Part/SKU,Part/Title")
       .Where(p => p.Part.SKU.StartsWith(SearchSku)).OrderBy(p => p.Part.SKU);

In this case, only the BinNumber, Quantity, Title, ID, and PartId values are retrieved from each inventory location item, and only SKU and Title are retrieved from each part item. If you use view projections, you need to be aware that the result set will include null values for the fields that you have omitted. For example, if you attempt to accessinventoryItem.Part.Description from a returned result, the value will always be null because your query excluded thePart.Description property. The expression results in the following REST query.

http://contoso/sites/sharepointlist/_vti_bin/listdata.svc/InventoryLocations()
  ?$filter=startswith(Part/SKU,'sku')
    &$orderby=Part/SKU
      &$expand=Part
        &$select=BinNumber,Quantity,Title,Id,PartId,Part/SKU,Part/Title

In addition to projecting fields from related entities onto a target entity—such as projecting Part fields onto an Inventory Location entity, as illustrated in the preceding example—you can also create a new view entity that combines the fields of interest. The following query populates a PartInvView instance that contains fields from the Inventory Locations list and the Parts list.

C#

var query = 
  (DataServiceQuery)context.InventoryLocations
    .Where(p => p.Part.SKU.StartsWith(SearchSku))
    .OrderBy(p => p.Part.SKU)
    .Select((i) => new PartInvView
       {
         BinNumber = i.BinNumber, 
         Quantity=i.Quantity, 
         InvTitle=i.Title,
         InvId=i.Id,
         PartId=i.PartId, 
         SKU=i.Part.SKU, 
         PartTitle=i.Part.Title 
       });

This projection produces the same REST query as the previous example. The only difference is that the service proxy will use the results to populate a collection of PartInvView instances, instead of a collection of InventoryLocationsIteminstances.

http://contoso/sites/sharepointlist/_vti_bin/listdata.svc/InventoryLocations()
  ?$filter=startswith(Part/SKU,'sku')
    &$orderby=Part/SKU
      &$expand=Part
        &$select=BinNumber,Quantity,Title,Id,PartId,Part/SKU,Part/Title

You should consider using view projections whenever you are retrieving lists of items, in order to reduce network traffic and improve query efficiency.

Concurrency Management

By default, the REST implementation supports optimistic concurrency. This means that no locks are placed on the underlying database tables between the time you read an item and the time you write an update to the item. This is a standard approach to service development that prevents clients from controlling precious database resources and impacting other clients. To detect whether an underlying entity has changed between a read operation and an update operation, the REST interface records information about the version of the entity you originally retrieved. If this version information has changed when you perform the update operation, the REST interface will return the following error.

XML

xml version="1.0" encoding="utf-8" standalone="yes"?><error xmlns="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata">  <code></code>  <message xml:lang="en-US">
    Since entity type 'Microsoft.SharePoint.DataService.PartsItem' has one or  
    more ETag properties, If-Match HTTP header must be specified for DELETE/PUT 
    operations on this type.
  </message></error>

The OData standard used by the REST interface uses ETags to perform this concurrency control. ETags are a mechanism defined by the HTTP protocol for efficient cache control between a client browser and a Web server. An ETag consists of a unique value that the Web server specifies to identify a particular version of a resource. When you update an entity, the service proxy will automatically add an ETag to the HTTP request. The ETag value matches the value provided by the service when you retrieved the data that you want to update. However, if the server-side data changes between the point at which you retrieve it and the point at which you persist an update, the ETag values will not match, and the server will detect a conflict. In this case, you receive the error described earlier. This error may also occur within your code if you have more than one data context retrieving the same entity, or if you create a new data context to save an item that was previously retrieved. If you want to persist your changes regardless of whether the underlying entity has changed, you can use the following code to force the service to apply your updates.

C#

context.MergeOption = MergeOption.OverwriteChanges;context.AttachTo("Parts", currentItem, "*");

The DataServiceContext.AttachTo method instructs the context object to track the object that you intend to update. By specifying an ETag value of *, you are telling the service to overwrite the object, regardless of the ETag value.

Note:
For more information, see Section 3.1, "Concurrency control and ETags," in OData: Operations.

PUT and MERGE Operations

The WCF Data Services proxy uses two different HTTP verbs for different update operations:

• A PUT request is used to update an entire entity. If no values are specified for fields in the entity, the fields will be set to default values.
• A MERGE request is used to update only those field values that have changed. Any fields that are not specified by the operation will remain set to their current value.

Because the service proxy and the DataServiceContext class manage the creation of HTTP requests, you generally do not need to worry about these details when you use the REST interface from managed code. However, when you use JavaScript, you must manually create the HTTP requests and, as such, you need to understand this distinction. The next section provides more details about using the REST interface from JavaScript.

Using the REST Interface from JavaScript

Using the REST interface from JavaScript requires some extra work, because you can't generate a service proxy to build requests and handle responses. In order to use the REST interface to create entities from JavaScript, you must perform the following actions:

• Create an HTTP request using the POST verb.
• Use the service URL of the list to which you want to add an entity as the target for the POST.
• Set the content type to application/json.
• Serialize the JSON objects that represent your new list items as a string, and add this value to the request body.

This is illustrated by the following code, which creates a new inventory location item. This simplified example was taken from the RestScripts.js file in the Client Reference Implementation.

JavaScript

var url = 'http://localhost/sites/sharepointlist/_vti_bin/listdata.svc/InventoryLocations';
var inventoryLocation = {};

// Insert a new Part location.
inventoryLocation.PartId = $('#hidPartId').val();
inventoryLocation.BinNumber = $('#binText').val();
inventoryLocation.Quantity = $('#quantityText').val();

var body = Sys.Serialization.JavaScriptSerializer.serialize(inventoryLocation);

$.ajax({
         type: 'POST',
         url: url,
         contentType: 'application/json',
         processData: false,
         data: body,
         success: function () 
         {
           alert('Inventory Location Saved.');
         }
       });

Updating an existing entity is a little more complex. If you've worked with REST services before, you might be tempted to use an HTTP PUT operation to update the entity. However, this approach can be problematic. Even if you load the entire entity, keep the entity in memory, and use the entity in a PUT operation, you may still experience problems with field values. Experience with this approach has shown issues with date time conversion and the population of lookup fields. This is because the OData protocol assumes that a PUT operation will update the entire entity, and any fields that are not explicitly specified are reset to their default values, most of which are a null value. A better approach is to use the HTTP MERGE operation, which updates only the fields that have changed. This approach also improves performance, because you don't need to initially retrieve a full representation of the entity just to send it back to the server to update it.

To use this approach to update an existing entity, you must perform the following actions:

• Create an HTTP request using the POST verb.
• Add an X-HTTP-Method header with a value of MERGE.
• Use the service URL of the list item you want to update as the target for the POST—for example,_vti_bin/listdata.svc/InventoryLocations(XXX), where XXX is the ID of the list item.
• Add an If-Match header with a value of the entity's original ETag.

This is illustrated by the following code, which updates an existing inventory location item. This simplified example was taken from the RestScripts.js file in the Client Reference Implementation.

C#

var locationId = $('#hidLocationId').val();
var url = 'http://localhost/sites/sharepointlist/_vti_bin/listdata.svc/InventoryLocations';
var beforeSendFunction;
var inventoryLocationModifications = {};

// Update the existing Part location.
url = url + "(" + locationId + ")";
beforeSendFunction = function (xhr) 
{
  xhr.setRequestHeader("If-Match", inventoryLocation.__metadata.ETag);
  // Using MERGE so that the entire entity doesn't need to be sent over the wire. 
  xhr.setRequestHeader("X-HTTP-Method", 'MERGE');
}

inventoryLocationModifications.BinNumber = $('#binText').val();
inventoryLocationModifications.Quantity = $('#quantityText').val();

var body = Sys.Serialization.JavaScriptSerializer.serialize(inventoryLocationModifications);

$.ajax({
         type: 'POST',
         url: url,
         contentType: 'application/json',
         processData: false,
         beforeSend: beforeSendFunction,
         data: body,
         success: function () 
         {
           alert('Inventory Location Saved.');
         }
       });

For more information about update and merge operations, see Section 2.6, "Updating Entries," in OData: Operations.

Batching

The OData protocol used by WCF Data Services supports the batching of multiple REST queries into a single HTTP request. Using batching reduces chattiness, uses network bandwidth more efficiently, and improves the responsiveness of your applications. In order to use batching, you simply submit multiple queries at the same time using theDataServiceContext.BeginExecuteBatch method.

C#

context.BeginExecuteBatch(DisplayParts, context, invQuery, partsQuery);

In this example, two queries are submitted: invQuery and partsQuery. The list of queries submitted is variable, so while this example shows two queries, additional queries could be added. When the server finishes executing a batch of requests, it returns a collection of results to the client. This is illustrated by the following code example.

C#

// Get the batch response.
DataServiceResponse Response = context.EndExecuteBatch(result);

// Loop through each operation.
foreach (QueryOperationResponse operation in Response)
{
  if (operation.Error != null)
  {
    throw operation.Error;
  }
  if (oOperation is QueryOperationResponse)
  {
    ProcessInventoryLocation(operation);
  }
  if (operation is QueryOperationResponse)
  {
    ProcessParts(operation);
  }
}

The service proxy sends batch requests in a multi-part message (MIME) format to the REST service. Notice that the message contains two GET requests, one for Inventory Locations and one for Parts.

HTTP Request

POST http://contoso/sites/sharepointlist/_vti_bin/listdata.svc/$batch HTTP/1.1
Content-Type: multipart/mixed; boundary=batch_16c7085d-ad1e-4962-b5e3-e7c83452b95a
Accept-Language: en-US
Referer: file:///C:/spg3/Trunk/Source/Client/Client.REST/Client.REST.Silverlight/Bin/Debug/Client.REST.Silverlight.xap
Authorization: Negotiate oXcwdaADCgEBoloEWE5UTE1TU1AAAwAAAAAAAABYAAAAAAAAAFgAAAAAAAAAWAAAAAAAAABYAAAAAAAAAFgAAAAAAAAAWAAAABXCiOIGAbAdAAAAD4N0FBUwhwapfSA5hPbF5jGjEgQQAQAAAPUXp1AtIpqEAAAAAA==
Accept-Encoding: identity
DataServiceVersion: 1.0;NetFx
MaxDataServiceVersion: 2.0;NetFx
Accept: application/atom+xml,application/xml
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.3; MS-RTC LM 8; .NET4.0C; .NET4.0E)
Host: contoso
Connection: Keep-Alive
Pragma: no-cache
Content-Length: 717

--batch_16c7085d-ad1e-4962-b5e3-e7c83452b95a
Content-Type: application/http
Content-Transfer-Encoding: binary

GET http://contoso/sites/sharepointlist/_vti_bin/listdata.svc/InventoryLocations()?$filter=startswith(Part/SKU,'sku11')&$orderby=Part/SKU&$expand=Part&$select=BinNumber,Quantity,Title,Id,PartId,Part/SKU,Part/Title HTTP/1.1
DataServiceVersion: 2.0;NetFx

--batch_16c7085d-ad1e-4962-b5e3-e7c83452b95a
Content-Type: application/http
Content-Transfer-Encoding: binary

GET http://contoso/sites/sharepointlist/_vti_bin/listdata.svc/Parts()?$filter=startswith(SKU,'sku11')&$select=Title,SKU,Id,Description HTTP/1.1
DataServiceVersion: 2.0;NetFx

--batch_16c7085d-ad1e-4962-b5e3-e7c83452b95a--

The response to the batch execution also uses MIME formatting, and it contains two HTTP responses, one for each query submitted.

Http Response

HTTP/1.1 202 Accepted
Cache-Control: no-cache
Content-Type: multipart/mixed; boundary=batchresponse_8ad6352b-ac02-4946-afc5-1df735bb7f55
Server: Microsoft-IIS/7.5
SPRequestGuid: 5f0f516c-78cf-4ffe-b37e-1c9e7168ef18
Set-Cookie: WSS_KeepSessionAuthenticated={0a9aa553-ad9a-401f-862a-2566fe4c94f4}; path=/
X-SharePointHealthScore: 0
DataServiceVersion: 1.0;
X-AspNet-Version: 2.0.50727
WWW-Authenticate: Negotiate oRswGaADCgEAoxIEEAEAAABDh+CIwTbjqQAAAAA=
Persistent-Auth: true
X-Powered-By: ASP.NET
MicrosoftSharePointTeamServices: 14.0.0.4762
Date: Sun, 30 May 2010 16:34:19 GMT
Content-Length: 4204

--batchresponse_8ad6352b-ac02-4946-afc5-1df735bb7f55
Content-Type: application/http
Content-Transfer-Encoding: binary

HTTP/1.1 200 OK
Cache-Control: no-cache
DataServiceVersion: 2.0;
Content-Type: application/atom+xml;charset=utf-8



  
  http://contoso/sites/sharepointlist/_vti_bin/listdata.svc/InventoryLocations
  2010-05-30T16:34:19Z
  
  
    http://contoso/sites/sharepointlist/_vti_bin/listdata.svc/InventoryLocations(18)
    
    2010-05-21T14:06:13-04:00
    
      
    
    
    
      
        
          http://contoso/sites/sharepointlist/_vti_bin/listdata.svc/Parts(12)
          
          2010-05-21T14:06:13-04:00
          
            
          
          
          
          
            
              LOCK WASHERS, 1/2 11
              SKU11
            
          
        
      
    
    
    
      
        
        12
        Bin 0.5.17
        9
        18
      
    
  

--batchresponse_8ad6352b-ac02-4946-afc5-1df735bb7f55
Content-Type: application/http
Content-Transfer-Encoding: binary

HTTP/1.1 200 OK
Cache-Control: no-cache
DataServiceVersion: 2.0;
Content-Type: application/atom+xml;charset=utf-8



  
  http://contoso/sites/sharepointlist/_vti_bin/listdata.svc/Parts
  2010-05-30T16:34:19Z
  
  
    http://contoso/sites/sharepointlist/_vti_bin/listdata.svc/Parts(12)
    
    2010-05-21T14:06:13-04:00
    
      
    
    
    
    
      
        LOCK WASHERS, 1/2 11
        SKU11
        
        12
      
    
  

--batchresponse_8ad6352b-ac02-4946-afc5-1df735bb7f55--

Note:
For more information about how the OData protocol implements batching, see OData: Batch Processing.

Synchronous and Asynchronous Operations

The service proxy for the SharePoint REST interface supports synchronous and asynchronous calls to the service. The approach to managing asynchronous operations is almost identical to the CSOM experience. As described in the section, "Using the Service Proxy," earlier in this topic, the DataServiceQuery class provides aBeginExecute method that you can use to asynchronously invoke a REST query. You should use the asynchronous approach in applications where you need to avoid blocking the UI thread. You cannot make a synchronous call to the REST interface from Silverlight or from JavaScript.

Using the Client Object Model

The client-side object model (CSOM) provides client-side applications with access to a subset of the SharePoint Foundation server object model, including core objects such as site collections, sites, lists, and list items. As described inData Access for Client Applications, the CSOM actually consists of three distinct APIs—the ECMAScript object model, the Silverlight client object model, and the .NET managed client object model—that target distinct client platforms. The ECMAScript object model and the Silverlight client object model provide a smaller subset of functionality. This is designed to enhance the user experience, because it minimize the time it takes Silverlight applications or JavaScript functions running in a Web page to load the files required for operation. The .NET managed client object model provides a larger subset of functionality for standalone client applications. However, these APIs provide a broadly similar developer experience and work in a similar way.

Note:
For more information about the CSOM, see Managed Client Object Model on MSDN. For examples of how to perform common tasks with the CSOM, see Common Programming Tasks on MSDN.

Of the three principal approaches to client-side data access, using the CSOM APIs is the only approach that provides the kind of hierarchical, strongly-typed representation of SharePoint objects, such as sites and Webs, that compares to server-side development. The CSOM is the only approach for any client-side data access scenarios beyond list queries. The CSOM allows you to query SharePoint lists by creating CAML queries. This is the most efficient way to query lists, although it requires that developers revert to creating CAML queries. Specific cases where you should favor the use of the CSOM include the following:

• You need to perform advanced list operations, such as complicated joins or paging. You can also perform joins through REST requests, although this is subject to various limitations.
• You need to manipulate SharePoint objects, such as sites or Webs.
• You need client-side access to other areas of SharePoint functionality, such as security.

Query Efficiency and the Client Object Model

Although the CSOM APIs mirror the server APIs in terms of functionality, the way CSOM actually works necessitates some changes to the way in which you approach your development tasks. The CSOM uses a set of specialized Windows Communication Foundation (WCF) services to communicate with the SharePoint server. Each API is optimized to work efficiently with remote clients and with the asynchronous loading model used by the Ajax and Silverlight frameworks. This section describes how these efficiency mechanisms work and how you can use them to best effect in your client-side SharePoint applications.

Request Batching

All the CSOM APIs include a ClientContext class that manages the interaction between client-side application code and the SharePoint server. Before you perform any operations in client-side code, you must instantiate a ClientContextobject with the URL of a SharePoint site, as shown by the following code example.

C#

ClientContext clientContext = new ClientContext(webUrl);

The clientContext instance provides programmatic access to the objects within your site, such as the current Webobject, the parent Site object, and a Lists collection. Communication with the server occurs when you call theExecuteQuery method, or the ExecuteQueryAsync method, on the ClientContext instance. Consider the following example, which was adapted from the Client Reference Implementation.

Notice that the class names in the CSOM differ from their server-side counterparts in that they no longer have the SP prefix, like SPList or SPWeb—instead, they are simply List and Web.

C#

private void GetParts(string searchSku)
{
  Parts.Clear();
  List partsList = clientContext.Web.Lists.GetByTitle("Parts");
  List inventoryLocationsList = 
    clientContext.Web.Lists.GetByTitle("Inventory Locations");

  CamlQuery camlQueryPartsList = new CamlQuery();
  camlQueryPartsList.ViewXml =
    @"
        
          
            
              
              " + searchSku + @"

"; CamlQuery camlQueryInvLocationList = new CamlQuery(); camlQueryInvLocationList.ViewXml = @" " + searchSku + @" "; partListItems = partsList.GetItems(camlQueryPartsList); inventoryLocationListItems = inventoryLocationsList.GetItems(camlQueryInvLocationList); clientContext.Load(partListItems); clientContext.Load(inventoryLocationListItems); clientContext.ExecuteQueryAsync(onQuerySucceeded, onQueryFailed); }

The following actions take place within this code example:

1. The client-side code uses the ClientContext class to define a series of operations to execute against a SharePoint site. In this example, the operations are the following:
   1. Retrieve the Parts list.
   2. Retrieve the Inventory Locations list.
   3. Build a query for the Parts list.
   4. Build a query for the Inventory Locations list.
   5. Execute the query against the Parts list.
   6. Execute the query against the Inventory Locations list.
   7. Load the Parts query results (which causes them to be returned to the client).
   8. Load the Inventory Locations query results.
2. The client code calls the ClientContext.ExecuteQueryAsync method. This instructs the CSOM to send a request containing all operations to the server.
3. The SharePoint server executes the series of operations in order and returns the results to the client.
4. The CSOM notifies the client-side code of the results by invoking the callback method associated with theonQuerySucceed delegate.

This following illustration shows this process.

Client object model request batching

This request batching process helps to improve performance and reduce network traffic in two ways. First, fewer Web service calls occur between the client and the SharePoint server, which reduces the "chattiness" of the client-server interface. For example, you can perform two list queries in a single request. Second, as a set of operations occur on the server in a single request, the data being acted on doesn't need to be moved between the client and the server for the intermediate operations—only the list of operations and the final result set are passed between the client and the server.

Request batching requires a different mindset when you create queries from client-side code. First, be aware that you do not have access to any results until you call ExecuteQueryAsync (or ExecuteQuery) and receive the call back with the results. If you need to implement conditional logic in the client-side code that can't be expressed in the command list that you send to the server, you will need to execute multiple queries. Second, you should aim to group your operations to minimize the number of service calls. This means you may need to think about how you sequence your logic in order to take full advantage of request batching.

List Queries with CAML

As you can see from the preceding code example, the CSOM supports querying list data using CAML. The CSOM allows you to submit any CAML queries that you could use with the SPQuery class, including the use of join statements and view projections. In fact, the CSOM is the only client-side data access mechanism that supports the use of joins with view projections that executes as a single list query.

Note:
The REST interface provides partial support for joins, when you specify implicit joins as part of the filter criteria. For more information, see Using the REST Interface. For an explanation of list joins and projections, see Data Access in SharePoint 2010.

The ability to use join statements and view projections in list queries leads to more efficient queries and reduces the number of queries required. The following example, which was adapted from the Client Reference Implementation, includes a CAML query that uses list joins and view projections. The query performs a left join between the Part Suppliers list and the Suppliers list. These lists are related by lookup columns, as shown in the following illustration.

Entity-relationship diagram

The query then uses a view projection to select the supplier name, DUNS, and rating that match a specified part SKU from the join table.

C#

private void GetPartSuppliers()
{
  if (currentItem != null)
  {
    List partSuppliersList = clientContext.Web.Lists.GetByTitle("Part Suppliers");
    CamlQuery camlQuery = new CamlQuery();
    camlQuery.ViewXml =
      @"
          
            
              
                
                " + currentItem.Part.Id + @"

"; partSuppliersListItems = partSuppliersList.GetItems(camlQuery); clientContext.Load(partSuppliersListItems); //Get Supplier Data clientContext.ExecuteQueryAsync(onPartSupplierQuerySucceeded, onQueryFailed); } }

In this example, the use of a list join dramatically improves the efficiency of the query and reduces network traffic. Without the list join, you would need to issue more queries and perform the join logic in your application code. The use of a view projection reduces the amount of data returned by the query, because it returns only a subset of field values that are relevant to your requirements. In the case of client-side data access, the benefits of this approach are even more pronounced. The ability to join lists in client-side data queries reduces the load on the server, reduces the number of round trips required between the client and the server, and reduces the overall amount of data transmitted between the client and the server.

The CSOM does not provide a mechanism for querying data across multiple lists that are not associated by a lookup field. In other words, there is no client-side functional equivalent of the SPSiteDataQuery class. If you need to perform a cross-list query from client-side code, consider creating a list view on the server that performs the list aggregation. You can then query the aggregated data from your client-side code.

Using LINQ for Objects

When you use the CSOM, you can write LINQ queries against client-side objects, such as lists and Webs, and then use the ClientContext class to submit these queries to the server. It's important to understand that when you take this approach, you are using LINQ to Objects to query SharePoint objects, not LINQ to SharePoint. This means that your LINQ expressions are not converted to CAML and you will not see the performance benefits that are associated with CAML conversion.

Note:
The LINQ to SharePoint provider is not supported by any of the client-side data access mechanisms.

Submitting a LINQ to Objects expression to the server reduces network traffic between the client and the server and alleviates some of the processing burden on the client platform. This is because you use the LINQ expression to narrow down the objects returned through server-side processing, instead of retrieving all objects in a collection, such as all Webs on a site and iterating through the collection on the client. LINQ to Objects makes it easy to specify fairly complex criteria with which to narrow down your result set. For example, you could use the following code to retrieve all non-hidden lists that have been created since March 20, 2010.

C#

private IEnumerable newLists;

var dt = new DateTime(2010, 3, 20);
var query = from list
            in clientContext.Web.Lists
            where list.Created > dt && list.Hidden == false
            select list;

newLists = clientContext.LoadQuery(query);
clientContext.ExecuteQueryAsync(onPartQuerySucceeded, onPartQueryFailed);

In-Place Load and Queryable Load

The client object model provides two different mechanisms for data retrieval—in-place load and queryable load.

• In-place load. This loads an entire collection into the client context. To perform an in-place load, you use theClientContext.Load method.
• Queryable load. This returns an enumerable collection of results. To perform a queryable load, you use theClientContext.LoadQuery method.

For example, the following code uses an in-place load to load the collection of lists in the context site into the client context object.

C#

clientContext.Load(clientContext.Web.Lists);
clientContext.ExecuteQueryAsync(onQuerySucceeded, onQueryFailed);

After executing the query, you can access the list collection through the clientContext.Web.Lists property. When you perform an in-place load, the client context manages object identity for you. If you modify a setting such as the title of a list, and then you perform a second query that loads the same list, the client context understands that the returned items refer to the same list and it preserves the changes.

The following code uses an equivalent queryable load to load the collection of lists in the context site.

C#

private IEnumerable allLists;
 
var query = from list in clientContext.WebLists
            select list;

this.allLists = clientContext.LoadQuery(query);
clientContext.ExecuteQueryAsync(onQuerySucceeded, on QueryFailed);

When you use a queryable load, you are not loading items into the client context. Instead, you are loading items into a results array—in this case, the allLists field. In this case, object identity is not managed by the client context. If you were to repeat the query, the client context would simply repopulate the allLists field from server-side data and would overwrite any changes you had made on the client in the meantime.

In terms of performance, there are no advantages or disadvantages to either approach. Because you can only use an in-place load to load one collection of objects at a time, there are circumstances in which you may want to use a queryable load to simultaneously load an alternative view of the data on your site. For example, suppose you would like to add the completion date for every project within your organization into all the calendars on your SharePoint site. The projects are distributed across several custom lists. In this scenario, you would use the in-place load for the collection of calendar lists, because these are the objects that you want to update. You would use the queryable load for the collection of project lists, because these will not be updated.

Note:
For more information about using the CSOM to load object collections and query lists, see Data Retrieval Overviewon MSDN.

Synchronous and Asynchronous Operations

As described earlier in this topic, the CSOM batches your client-side operations and sends them to the server for execution. The CSOM provides both a synchronous model and an asynchronous model for invoking this server-side execution. If you use the .NET managed client API or the Silverlight client API, you can use either approach. If you use the ECMAScript API, you must use the asynchronous approach. This prevents service calls from causing the Web browser to block user interaction for the duration of the call.

For example, suppose you are using the .NET managed client API or the Silverlight client API to retrieve data from the server:

• If you call the ClientContext.ExecuteQuery method, your operation will be invoked synchronously. The thread that executes your code will wait for the server to respond before continuing.
• If you call the ClientContext.ExecuteQueryAsync method, your operation will be invoked asynchronously. In this case, you specify callback methods to handle the server response, and the current thread remains unblocked.

Although the Silverlight client API supports the synchronous ExecuteQuery method, in most cases you will want to useExecuteQueryAsync to submit your operation set. The following example, taken from the Client Reference Implementation, illustrates how you can use the ExecuteQueryAsync method with the Silverlight client API. ThePartSearchButton_Click method executes when the user clicks a button in the Silverlight application.

C#

private void PartSearchButton_Click(object sender, RoutedEventArgs e)
{
  bindingViewsModels.Clear();
  List partsList = clientContext.Web.Lists.GetByTitle("Parts");
            
  CamlQuery camlQueryPartsList = new CamlQuery();
  camlQueryPartsList.ViewXml = @"
    
       
       
         
           
           " + PartSkuTextBox.Text + @"

"; partListItems = partsList.GetItems(camlQueryPartsList); clientContext.Load(partListItems); clientContext.ExecuteQueryAsync(onQuerySucceeded, onQueryFailed); }

The ExecuteQueryAsync method accepts two arguments—a delegate for a method that is called if the server-side operation succeeds and a delegate for a method that is called if the server-side operation fails. If the operation is successful, the onQuerySucceeded method is called.

C#

private void onQuerySucceeded(object sender, ClientRequestSucceededEventArgs args)
{
  this.Dispatcher.BeginInvoke(DisplayParts);
}

As you can see, this method also makes an asynchronous method call. The Dispatcher.BeginInvoke method invokes the DisplayParts method on the user interface (UI) thread. This is a mandatory approach when you work with Silverlight, because you must use the UI thread to execute logic that updates the UI. The DisplayParts method simply binds the query results to the appropriate UI controls. The following illustration shows this overall process.

Asynchronous execution with the Silverlight client API

In the previous example, what would happen if you called ExecuteQuery instead of ExecuteQueryAsync? Silverlight would throw an InvalidOperationException with the following message:

The method or property that is called may block the UI thread and it is not allowed. Please use background thread to invoke the method or property, for example, using System.Threading.ThreadPool.QueueUserWorkItem method to invoke the method or property.

In other words, Silverlight will not allow you to block the UI thread. To avoid this exception, you would need to execute the query on a background thread, as shown in the following code example.

C#

private void PartSearchButton_Click(object sender, RoutedEventArgs e)
{
  bindingViewsModels.Clear();
  List partsList = clientContext.Web.Lists.GetByTitle("Parts");
  List inventoryLocationsList = 
    clientContext.Web.Lists.GetByTitle("Inventory Locations");
            
  CamlQuery camlQueryPartsList = new CamlQuery();
  camlQueryPartsList.ViewXml = @"
    
      
        
          
            
            " + PartSkuTextBox.Text + @"

"; partListItems = partsList.GetItems(camlQueryPartsList); clientContext.Load(partListItems); System.Threading.ThreadPool.QueueUserWorkItem( new WaitCallback(ThreadCallback), clientContext); } private void ThreadCallback(object s) { var context = (ClientContext)s; context.ExecuteQuery(); this.Dispatcher.BeginInvoke(DisplayParts); }

In other words, if you don't use the ExecuteQueryAsync method, you must manually implement the asynchronous logic. Both methods are functionally correct. However, the ExecuteQueryAsync method makes your code easier to understand and is preferred from a stylistic perspective. The ExecuteQuery method is useful in applications where a synchronous execution model is appropriate, such as a command-line application or a PowerShell extension.

Accessing Binary Data

In some cases, you may want to either upload or download binary files, such as images or documents, from your Rich Interactive Application (RIA) or managed client. You can use the CSOM to access binary data like documents; however, there are varying levels of support, depending on the CSOM API that you use.

The managed version of the CSOM supports both binary file upload and download using File.OpenBinaryDirect andFile.SaveBinaryDirect. The following example shows how to retrieve and save a picture using these APIs from a Windows Presentation Foundation (WPF) client. In this case, the method DisplayPix retrieves all the items in the Pix picture library and displays them to a WPF control in the call to AddPix. SavePix saves the file stream provided as an image to the Pix library, using the file name for the title.

C#

private void DisplayPix()
{
  List pixList = clientContext.Web.Lists.GetByTitle("Pix");

  CamlQuery camlQuery = new CamlQuery();
  camlQuery.ViewXml = "";
  pictureItems = pixList.GetItems(camlQuery);

  clientContext.Load(pictureItems);
  clientContext.ExecuteQuery();

  foreach (ListItem pictureItem in pictureItems)
  {
    string fileRef = pictureItem["FileRef"].ToString();
    FileInformation fileInfo = File.OpenBinaryDirect(clientContext, fileRef);
    AddPix(fileInfo.Stream);
  }
}

private void SavePix(string filename, Stream stream)
{
  string url = "/sites/sharepointlist/Pix/" + filename;
  File.SaveBinaryDirect(clientContext, url, stream, true);
}

For a detailed example of using this API with the managed CLOM, see Using the SharePoint Foundation 2010 Managed Client Object Model with the Open XML SDK 2.0 on MSDN.

The Silverlight CSOM supports opening binary files from SharePoint with File.OpenBinaryDirect using the same syntax as the managed CSOM but does not support saving binary files. The European Computer Manufacturers Association (ECMA) script (JavaScript) CSOM does not support either saving or reading binary files from SharePoint.