Tuesday, September 29, 2015

Running SharePoint 2010 on Windows 8 Consumer Preview

If you're a SharePoint geek and you love beta software (just like me) then you'll want to know if it's possible to set up SharePoint 2010 on the Windows 8 Consumer Preview.  I'm happy to report that it is, and that it's relatively easy.  :)
What you need to do is install all of the prerequisites manually, turn UAC off and then follow these instructions for getting it set up on Windows 7: http://msdn.microsoft.com/en-us/library/ee554869.aspx
You do need to be careful about the Windows Features though, as the package manager script listed on MSDN will not install the necessary .NET 3.5 features:
 
Before you can run up Central Administration, or any other SharePoint sites however, you will need to change the .NET Framework version to v2.0
Open up IIS Manager by hitting start and then typing inetmgr then hit enter.  You then need to expand application pools and right click on the app pool you want to change (all SharePoint App Pools will need to run under version 2.0.50727) and click basic settings:
IIS Manger
Once there, change the app pool from v4.0 to v2.0:
 After you've got it all up and running, you can then create SharePoint projects in Visual Studio, and use it for prototyping and playing around.  I still haven't managed to get the SharePoint PowerShell cmdlets to work as yet, since PowerShell runs against the v4 framework.  If anyone has any hints about this, then please let me know!

Installing SharePoint Server 2010 on Windows 8

If you want to install SharePoint Server 2010 in Windows 8, you need to read what I did to achieve it.
1. Follow the exact steps in this article->  http://msdn.microsoft.com/en-us/library/ee554869.aspx except:
You are using Windows 8 and can't install Windows identity Foundation, SQL Server Native Client and 2 other hotfixes that are for Windows 7.
Why?
Identity Foundation is a Windows 8 feature and can be installed from a Windows 8 disk.

SQL Server Native Client won't be needed because you'll be installing SQL Server before these steps.

2. Other Hotfixes are for Windows 7 and Windows Server 2008 R2. They can't be installed even if you wanted to.
After that a restart might be needed. 
2. Install SharePoint Server 2010.
3. The installation must be successful. Then first you need to edit some values in web.config inside wss.

Get to here: C:\inetpub\wwwroot\wss\VirtualDirectories\80\web.config (wherever SharePoint is installed, the port in my PC is 80)
And find these values and remove them:
 <section name="scriptResourceHandler"  type="System.Web.Configuration.ScriptingScriptResourceHandlerSection, System.Web.Extensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" requirePermission="false" allowDefinition="MachineToApplication" />
And this:
  <add name="TermStoreGenericObjectConverter" 

<jsonSerialization maxJsonLength="5000000">    <converters>
type="Microsoft.SharePoint.Taxonomy.WebServices.TermStoreGenericObjectConverter,
Microsoft.SharePoint.Taxonomy, Version=14.0.0.0,
Culture=neutral, PublicKeyToken=71e9bce111e9429c" />
    </converters>
</
jsonSerialization>


After that, get to the Internet Information Services application, and change the Application Pool of SharePoint - 80 (it's 80 in my machine):

Change the .NET Framework version to .NET Framework 2.0 and Recycle the Pool.
 

Exit the Internet Information Server application.
4. Run the SharePoint 2010 Products Configuration Wizard which will help you finalize your environment.

5. If you updated your SharePoint Server 2010 to SP1 through Windows Update then

you only need to update the second XML referring to JsonSerialization.

After that you need to reconfigure your SharePoint as we did in 5.
Since ASP.NET Permission isn't supported for Security purposes in SharePoint 2010 they do not advise working with ASP.NET 4.0 in SharePoint Server. That's why we configured it to use .NET Framework 2.0.

SharePoint Online Web Service Authentication using WCF Client-side behaviour

With the release SharePoint in 2013 and the ever increasing numbers taking up the SharePoint Online offering, it’s a good time to start looking at some of the challenges when moving to these platforms.
SharePoint has traditionally been a presentation technology with its own unique SharePoint development model utilising SharePoint designer and custom Web Part development. With the latest release, SharePoint 2013 that development model has been challenged by a new autonomous development model where the complexities and constraints of SharePoint as a development and deployment platform has been replaced by a service oriented integration platform for multiple independently hosted applications.
SharePoint has always supported a rich and expanding set of services with every release. As part of the move to the new development model a lot of work has gone into improving the “Client side” access into SharePoint by wrapping services in client side APIs and delivered some as easy to consume REST/oData endpoints, but the functionality of those services is still limited and has not yet evolved to match the power of the native SharePoint Server API nor the native SharePoint Web Services.
When moving from an on-premise SharePoint to SharePoint Online web service clients that were previously written against SharePoint Web Services (like UserProfile.asmx or Lists.asmx) will stop working because the authentication model has changed. A web service client that previously used Active Directory accounts to access SharePoint will need to become “claims aware” to use SharePoint Online. This involves syncing active directory accounts up to the MS Online Active Directory and authenticating at the MS Online Security Token Service (STS) before accessing SharePoint.
We hit this problem where a client had built a set of jobs, utilities and components that request and update SharePoint through services which then broke when migrating to SharePoint Online. The solution to updating these clients is well covered by this blog by Wictor Wilén. What this involves is opening up the source code for each client and inserting the appropriate authentication code using the MSOnlineClaimsHelper to do get the authentication cookies before calling the service.
The problem is we really didn’t want to pick through someone else’s code and find all the places to insert the authentication code. Surely there is an easier way to inject this boilerplate code without having to recode all the clients? Turns out there is, if those clients are written using WCF. What we can do is write a WCF Client side behaviour to detect when authentication needs to be done, make the appropriate requests to MS Online STS, attach the cookies and then allow the SharePoint service request proceed as normal.
To use the ClaimsHelper, first set up the Uris of the SharePoint resources you want to access and the corresponding username/passwords using a CredentialCache and set it to the MsOnlineClaimsHelper.CredentialCache.
            CredentialCache credentialCache = new CredentialCache();            credentialCache.Add(_hostUri, “Basic”new NetworkCredential(“peter.reid@kloud.com.au”“blah”));            MsOnlineClaimsHelper.CredentialCache = credentialCache;Then there are two ways to use this library. Either manually request the cookies or attach to the outgoing request like this:
            UserProfile.UserProfileServiceSoapClient userProfile = new UserProfile.UserProfileServiceSoapClient(“UserProfileServiceSoap”, _hostUri + “_vti_bin/userprofileservice.asmx”);            using (new OperationContextScope(userProfile.InnerChannel))            {                _helper.AddAuthCookies(OperationContext.Current);                UserProfile.PropertyData[] data = userProfile.GetUserProfileByName(userName);                return Guid.Parse(GetPropertyValue(data, “UserProfile_GUID”));            }Or automatically do the same by adding the WCF client side behaviour like this:
 
    <behaviors>       <endpointBehaviors>         <behavior name="CookieBehavior">           <CookieBehaviorExtension/>         </behavior>       </endpointBehaviors>     </behaviors>     <extensions>       <behaviorExtensions>         <add name="CookieBehaviorExtension" type="ClaimsHelper.CookieBehaviourExtension, ClaimsHelper, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/>       </behaviorExtensions>     </extensions>     <client>       <endpoint behaviorConfiguration="CookieBehavior" binding="basicHttpBinding" bindingConfiguration="BasicBinding"                contract="UserProfile.UserProfileServiceSoap" name="UserProfileServiceSoap" />     </client>Which enables you to use a standard web service client call knowing that the authentication conversation is happening under the hood by the WCF client side behaviour.
            UserProfile.UserProfileServiceSoapClient userProfile
                    = new UserProfile.UserProfileServiceSoapClient("UserProfileServiceSoap", _hostUri + "_vti_bin/userprofileservice.asmx");
            UserProfile.PropertyData[] data = userProfile.GetUserProfileByName(userName);
            return Guid.Parse(GetPropertyValue(data, "UserProfile_GUID"));
Check out the code

MsOnlineClaimsHelper vs. SharePointOnlineCredentials

When you want to use the CSOM (Client Side Object Model) with SharePoint Online, you currently have quite a few options on how to authenticate. For a SharePoint newcomer or even experienced SharePoint developers, it can be quite tricky to know when to use what, this post is an attempt to provide an overview.
The reason for this article was the SharePoint code published together with the MyShuttle example from Visual Studio Connect 2014, code can be found here. The code sample contains an Azure WebJob (MyShuttle.WebJob) that sends invoices to a SharePoint site, during the presentation of the code it’s mentioned that the sample was created with Office 365 API, but when the code was released that wasn’t the case. I also believe that the way they authenticate in the code is no longer best practice, which is why I wanted to publish this article.

How do CSOM authenticate

First things first, how does the CSOM actually authenticate when used with SharePoint online? It might not be obvious when you use the library, but everything you do in CSOM eventually turns into a REST call to SharePoint. So the CSOM library uses the REST API endpoints hosted at sharepointsite.sharepoint.com/_api/?
Hold on, not so quick, unfortunately that is not the case, because the CSOM was invented before the SharePoint product team decided that it was a good idea to have a open REST implementation. So all calls from CSOM end up at sharepointsite.sharepoint.com/_vti_bin/client.svc instead. At some point down the road we will hopefully see that the api endpoint will have the same functionality as the client.svc endpoint, and Microsoft will be able to change the implementation of the CSOM library to use the api endpoint, but that’s not where we are now.
Enough history lets get back to authentication, or in essence how do SharePoint know who we are when we call from CSOM? There are two options:
  • We have a fedauth authentication cookie, that we pass along with our request
  • We have an access token, that we pass along with our request in the http-header
Now, it might seem a little confusing that there are two different methods to authenticate your calls, but again it’s due to the history of SharePoint the fedauth cookie, is the old way of doing things, while the access token is the new school (OAuth2) way of doing it.
So how do you obtain either one of them? This is were it gets interesting, because depending on how you use the CSOM library you will end up with different methods of authentication. Currently there exists three practical ways to authenticate when using CSOM.
  • MsOnlineClaimsHelper (fedauth)
  • SharePointOnlineCredentials (fedauth)
  • Acccess Token (ADAL/Office 365 API)

MsOnlineClaimsHelper

In the early days of SharePoint Online development, some smart people wrote a class named MsOnlineClaimsHelper that could be used to get the the fedauth cookie that was needed to make authenticated requests to SharePoint online. This class have since been floating around on the internet in different versions. One of the first (if not the first) blog post about it was done by Wictor Willen who was very involved in it.
A different version of this MsOnlineClaimsHelper class is also the one that Microsoft used in their MyShuttle example.
Personally I don’t think you should be using this class for SharePoint online, because SharePointOnlineCredentials class maintained by Microsoft, and part of the Microsoft.SharePoint.Client.Runtime assembly does pretty much the same.
There might be some hybrid scenarios where you might still benefit from having access to change the source code, where MsOnlineClaimsHelper is the way to go, but for everyday SharePoint online use, you should go with SharePointOnlineCredentials.

SharePointOnlineCredentials

With release of SharePoint 2013, and the big push towards SharePoint online, Microsoft obviously needed a more frictionless way to have people use CSOM with SharePoint Online than before. Due to that they added SharePointOnlineCredentials to the client side framework, so people easily could provider username and password of a user account and use that users permissions when accessing SharePoint via. CSOM.
The simplest example would be something like this.
using Microsoft.SharePoint.Client;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace ConsoleApplication1
{
    class Program
    {
        static void Main(string[] args)
        {
            using (var context = new ClientContext("https://tenant.sharepoint.com/"))
            {
                var ss = new System.Security.SecureString();
                Array.ForEach("Pass@w0rd".ToCharArray(),(c) => { ss.AppendChar(c); });
                context.Credentials = new SharePointOnlineCredentials("admin@tenant.onmicrosoft.com", ss);
                var web = context.Web;
                context.Load(web);
                context.ExecuteQuery();
                var title = web.Title;
                Console.WriteLine(title);
            }
        }
    }
}

Acccess Token (ADAL/Office 365 API)

In 2014 we have seen Microsoft invest a lot in what they called the Office 365 API tools. These tools use the REST (_api) endpoint of SharePoint Online and gets authenticated by the access token in a header. The access token is coming from an Azure Active Directory Application that is registered to allow delegation of access to SharePoint. The level of access to SharePoint is always equal to the access level of the user executing the call.
Right now the functionality of the Office 365 API client libraries are rather limited (Microsoft.Office365.SharePoint) so if you want to do anything other than the most simple things you have to build the REST request yourself. That can be annoying and also as mentioned not everything can be accomplished by the REST endpoint. So if you want you are actually also able to use the acquired access token to authenticate with the CSOM, in that way the full client side functionality is available to use.
If you are interested in the Access Token approach, I encourage you to read Steve Peschka’s blog post on the topic.
One final note, when working with SharePoint apps, you are also using the Access Token approach. Hope this article helps clear up a few things.



The scenario is the following: I need to perform a federated authentication of a user (which uses his university account) into the Sharepoint site of his university and to obtain both the FedAuth and rtFa cookies (which I have to pass to SharePoint REST webservices in order to access resources).
I made some attempts but there is at least an issue in each one:

1) Using Microsoft.SharePoint.Client library

ClientContext context = new ClientContext(host);
SharePointOnlineCredentials creds = new SharePointOnlineCredentials(user, passw);
context.Credentials = creds;

Uri sharepointuri = new Uri(host);
string authCookie = creds.GetAuthenticationCookie(sharepointuri);

Web web = context.Web;
context.Load(web, w=>w.Lists);
context.ExecuteQuery();

fedAuthString = authCookie.Replace("SPOIDCRL=", string.Empty);
This way I manage to get the FedAuth cookie but I am unable to get the rtFa cookie.
How can I get the rtFa cookie at this point? Can I intercept the HTTP request involved in such an operation (i.e., context.ExecuteQuery()) -- which presumably contains the rtFa cookie in the headers? Or, can I get the rtFa cookie by only leveraging on the FedAuth cookie?

2) Using MsOnlineClaimsHelper

This is a helper class which can be found on the Internet (e.g., here http://blog.kloud.com.au/tag/msonlineclaimshelper/ ).
This class, as it is, works with normal authentication but fails with federated authentication.
So I adjusted it in order to make it work in this case. As long as I understand, the steps are the following:
  1. Authenticate using username and password to the STS ADFS service of the university (the "federated party" or the ISSUER) -- here the Relying Party is Sharepoint O365 STS ("https://login.microsoftonline.com/extSTS.srf")
  2. If the auth succeeds, I get back a SAML assertion containing the claims and a security token
  3. Now, I authenticate to the SharePoint site by passing the Security Token
  4. If the token is recognized, I get back a response which contains the two cookies (FedAuth and rtFa)
I am not an expert in this matter, and I came out with the following code:
This is the code that calls the method above and try to get FedAuth and rtFa from credentials in two steps (step 1: get SAML token from Federated Party; step 2: pass token from Federated Party to Sharepoint):
     private List GetCookies(){
            // 1: GET SAML XML FROM FEDERATED PARTY THE USER BELONGS TO
            string samlToken = getResponse_Federation(sts: "https://sts.FEDERATEDDOMAIN.com/adfs/services/trust/13/usernamemixed/",
                realm: "https://login.microsoftonline.com/extSTS.srf");

            // 2: PARSE THE SAML ASSERTION INTO A TOKEN 
            var handlers = FederatedAuthentication.ServiceConfiguration.SecurityTokenHandlers;
            SecurityToken token = handlers.ReadToken(new XmlTextReader(new StringReader(samlToken )));

            // 3: REQUEST A NEW TOKEN BASED ON THE ISSUED TOKEN
            GenericXmlSecurityToken secToken = GetO365BinaryTokenFromToken(token);

            // 4: NOW, EASY: I PARSE THE TOKEN AND EXTRACT FEDAUTH and RTFA
            ...............
    }


    private string getResponse_Federation(string stsUrl, string relyingPartyAddress)
    {
        var binding = new Microsoft.IdentityModel.Protocols.WSTrust.Bindings.UserNameWSTrustBinding(SecurityMode.TransportWithMessageCredential);
        binding.ClientCredentialType = HttpClientCredentialType.None;

        var factory = new WSTrustChannelFactory(binding,  stsUrl);

        factory.Credentials.UserName.UserName = "username";
        factory.Credentials.UserName.Password = "password";
        factory.Credentials.SupportInteractive = false;
        factory.TrustVersion = TrustVersion.WSTrust13;

        IWSTrustChannelContract channel = null;
        try
        {
            var rst = new RequestSecurityToken
            {
                RequestType = WSTrust13Constants.RequestTypes.Issue,
                AppliesTo = new EndpointAddress(relyingPartyAddress), //("urn:sharepoint:MYFEDERATEDPARTY"),
                ReplyTo = relyingPartyAddress,
                KeyType = WSTrust13Constants.KeyTypes.Bearer,
                TokenType =  "http://docs.oasis-open.org/wss/oasis-wss-saml-token-profile-1.1#SAMLV2.0",
                RequestDisplayToken = true,
            };
            channel = (WSTrustChannel)factory.CreateChannel();

            RequestSecurityTokenResponse response = null;
            SecurityToken st = channel.Issue(rst, out response);
            var genericToken = st as GenericXmlSecurityToken;
            return genericToken.TokenXml.OuterXml;
        }
        catch (Exception e)
        {
            return null;
        }
    }

    private GenericXmlSecurityToken GetO365BinaryTokenFromToken(SecurityToken issuedToken)
    {
        Uri u = new Uri("https://login.microsoftonline.com/extSTS.srf");

        WSHttpBinding binding = new WSHttpBinding(SecurityMode.TransportWithMessageCredential);
        binding.Security.Transport.ClientCredentialType = HttpClientCredentialType.None;
        binding.Security.Mode = SecurityMode.TransportWithMessageCredential;
        binding.Security.Message.ClientCredentialType = MessageCredentialType.IssuedToken;

        Microsoft.IdentityModel.Protocols.WSTrust.WSTrustChannelFactory channel =
        new Microsoft.IdentityModel.Protocols.WSTrust.WSTrustChannelFactory(
            binding, new EndpointAddress("https://login.microsoftonline.com/extSTS.srf"));

        channel.TrustVersion = TrustVersion.WSTrust13;
        channel.Credentials.SupportInteractive = false;

        GenericXmlSecurityToken token = null;

        try
        {
            RequestSecurityToken rst = new RequestSecurityToken(WSTrust13Constants.RequestTypes.Issue, WSTrust13Constants.KeyTypes.Bearer)
            {
            };
            rst.AppliesTo = new EndpointAddress("urn:sharepoint:MYFEDERATEDPARTY");
            channel.ConfigureChannelFactory();
            var chan = (Microsoft.IdentityModel.Protocols.WSTrust.WSTrustChannel)channel.CreateChannelWithIssuedToken(issuedToken);

            RequestSecurityTokenResponse rstr = null;

            token = chan.Issue(rst, out rstr) as GenericXmlSecurityToken;

            return token;
        }
        catch (Exception ex){
            Trace.TraceWarning("WebException in getO365BinaryTokenFromADFS: " + ex.ToString());
            throw;
        }
    }
I managed to get back a SAML token from the university STS. However, when parsed, the resulting SecurityToken has no security keys (i.e., the SecurityKeys collection is empty)
With no keys, I get on GetO365BinaryTokenFromToken() but when I try to send the token to the SharePoint Authentication service -- I get the following error: "The signing token Microsoft.IdentityModel.Tokens.Saml2.Saml2SecurityToken has no keys. The security token is used in a context that requires it to perform cryptographic operations, but the token contains no cryptographic keys. Either the token type does not support cryptographic operations, or the particular token instance does not contain cryptographic keys. Check your configuration to ensure that cryptographically disabled token types (for example, UserNameSecurityToken) are not specified in a context that requires cryptographic operations (for example, an endorsing supporting token)."
I think that there are also some configuration issues that I cannot control directly, on both sides (the university STS ADFS and the Sharepoint STS).
I hope that more expert people would bring clarity in this process and even provide advice to actually make this scenario work.

File download function

With the following function, I am able to download a file (given an URL such as https://myfederatedparty.sharepoint.com/sites/MYSITE/path/myfile.pdf) by issuing BOTH the FedAuth and the rtFa cookie. If I do not pass the rtFa cookie, I get an "Unauthorized" response.
    public static async Task<byte[]> TryRawWsCall(String url, string fedauth, string rtfa, CancellationToken ct, TimeSpan? timeout = null) {
        try {
            HttpClientHandler handler = new HttpClientHandler();
            handler.CookieContainer = new System.Net.CookieContainer();
            CookieCollection cc = new CookieCollection();
            cc.Add(new Cookie("FedAuth", fedauth));
            cc.Add(new Cookie("rtFa", rtfa));
            handler.CookieContainer.Add(new Uri(url), cc);

            HttpClient _client = new HttpClient(handler);
            if (timeout.HasValue)
                _client.Timeout = timeout.Value;
            ct.ThrowIfCancellationRequested();

            var resp = await _client.GetAsync(url);
            var result = await resp.Content.ReadAsByteArrayAsync();
            if (!resp.IsSuccessStatusCode)
                return null;
            return result;
        }
        catch (Exception) { return null; }
    }

View Tenant (ULS) Logs in SharePoint Online using CSOM

Even though the classes exist in the CSOM, the Microsoft Office 365 Engineering team has confirmed that this is something which is not poss...