Trying to download multiple files from sharepoint


  • I am trying to pull down 135K+ files from our SharePoint repository.  Needless to say it is slow and I wanted to wrap it in a Parallel.ForEach loop.  I understand the ClientContext is not thread safe.  So looking around researching the problem, I understand the problem so here is what I have done:

     public static List<SP.ListItem> GetAllDocumentsInaLibrary()
                List<SP.ListItem> items = new List<SP.ListItem>();
                string siteURL ="URL to Server";
                using (var ctx = new SP.ClientContext(siteURL))
                    ctx.Load(ctx.Web, a => a.Lists);
                    int i = 0;
                    SP.List list = ctx.Web.Lists.GetByTitle("ArchivedData");
                    SP.ListItemCollectionPosition position = null;
                    int rowLimit = 100;
                    var camlQuery = new SP.CamlQuery();
                    camlQuery.ViewXml = @"<View Scope='RecursiveAll'>
                                            <OrderBy Override='TRUE'><FieldRef Name='ID'/></OrderBy>
                                                <FieldRef Name='Title'/><FieldRef Name='ID' />
                                          <RowLimit Page='TRUE'>" + rowLimit + "</RowLimit></View>";
                        SP.ListItemCollection listItems = null;
                        camlQuery.ListItemCollectionPosition = position;
                        listItems = list.GetItems(camlQuery);
                        i += 100;
                        position = listItems.ListItemCollectionPosition;
                            () => CreateClientContext(siteURL),
                                (currentfileItem, localContext) =>
                                    localContext.Load(currentfileItem, x => x.File);
                                    var fileRef = currentfileItem.File.ServerRelativeUrl;
                                    var fileInfo = SP.File.OpenBinaryDirect(localContext, fileRef);
                                    var fileName = @"c:\downloads\" + currentfileItem.File.Name;
                                    using (var fileStream = System.IO.File.Create(fileName))
                                        //Generate MD5 HASH
                                        using (var md5 = MD5.Create())
                                            Console.WriteLine("MD5 Hash for file {0}: {1}", fileName, md5.ComputeHash(fileStream).GetHashCode());
                                            //Record HASH and File name in db with file name and title if there, and SharePointID
                                            using (SqlConnection sqlCon = new SqlConnection("[Connection String"))
                                                using (SqlCommand sqlCmd1 = new SqlCommand { CommandText = "INSERT INTO [TABLE] ([FileName], [FileTitle], [SharePointId], [MD5Hash]) VALUES (@fileName, @fileTitle, @sharePointId, @MD5Hash)", Connection = sqlCon })
                                                    sqlCmd1.Parameters.AddWithValue("@fileName", currentfileItem.File.Name);
                                                    sqlCmd1.Parameters.AddWithValue("@fileTitle", String.IsNullOrEmpty(currentfileItem.File.Title) ? "null" : currentfileItem.File.Title);
                                                    sqlCmd1.Parameters.AddWithValue("@sharePointId", currentfileItem.Id);
                                                    sqlCmd1.Parameters.AddWithValue("@MD5Hash", md5.ComputeHash(fileStream).GetHashCode().ToString());
                                    //Remove the file we don't need it anymore
                    while (position != null);
                return items;

    Here is the code for creating the clientContext

     private static ClientContext CreateClientContext( string url)
                ClientContext context = new ClientContext(url);
                return context;

    I am getting an error on the Parallel.ForEach loop trying to add the ClientContext to the loop:

     () => CreateClientContext(siteURL)

    It keeps coming up with this error:

    "Cannot convert lambda expression to type 'System.Threading.Tasks.ParallelOptions' because it is not a delegate type"

    I know what the error is saying, but I am having problems formulating the Parallel.ForEach.  Can someone help me on this?


    2018년 5월 18일 금요일 오후 3:21

모든 응답

  • Hi,

    You have instantiate a ClientContext by using (var ctx = new SP.ClientContext(siteURL)), may I know why you want to instantiate a new one in ForEach?           

    Best Regards,


    Please remember to mark the replies as answers if they helped. If you have feedback for TechNet Subscriber Support, contact

    Click here to learn more. Visit the dedicated forum to share, explore and talk to experts about Microsoft Teams.

    2시간 43분 전