sharepoint online – Querying the document library (containing more than 5,000 documents) with FolderServerRelativeUrl results in the error "Exceeds the threshold for the list view".

I have a document library that contains 6,000+ items, and I've written the following code into a remote event receiver to get the documents that are in a specific folder FolderServerRelativeUrl, as follows:-

ListItemCollectionPosition position = null;
 do
{
      CamlQuery camlQuery6 = new CamlQuery();
      camlQuery6.ViewXml = @"5000";//string.Format("{0}{1}", "0", "3000");
      camlQuery6.ListItemCollectionPosition = position;
      camlQuery6.FolderServerRelativeUrl = context.Web.ServerRelativeUrl + "/ArchDocs/" + currentFilingSystemItem("DealName").ToString();

      ListItemCollection collListItem6 = context.Web.GetList(context.Web.ServerRelativeUrl + "/ArchDocs").GetItems(camlQuery6);
      context.Load(collListItem6, items => items.Include(
                                                  item => item.Id,
                                                  item => item("FileDirRef"),
                                                  item => item("Title"),
                                                   ), items => items.ListItemCollectionPosition);
      context.ExecuteQuery();
      position = collListItem6.ListItemCollectionPosition;

              foreach (ListItem listItem in collListItem6) // collect the items we found in this chuck of searched items
                  {
                       foundListItems.Add(listItem);
                  }
}
while (position != null);
foreach (ListItem item in foundListItems)
                  {



                  }

but my code above will trigger this error even though I am using paging which should allow me to query large document libraries: –

The attempted operation is forbidden because it goes beyond the list view
Threshold enforced by the administrator.

Any advice on this please?
Thank you very much

Algorithms – Find the largest sum of $ k $ elements below the threshold

You can use meet-in-the-middle to reduce the runtime to $ O (n ^ { lceil k / 2 rceil}) $.

Let me assume for the sake of simplicity $ k $ is just.

The idea is as follows:

  • partition wall $ A $ in two parts.
  • Calculate a sorted list of sums of for each part $ k / $ 2 Elements from the part.
  • For each $ k / $ 2– sum up the binary search in the first part to find the best $ k / $ 2-sum in the second part, which corresponds to the restrictions.

As mentioned earlier, there are several problems with the idea:

  1. It is assumed that the optimal solution contains exactly $ k / $ 2 adds up from each part.
  2. It runs in time $ O (n ^ {k / 2} log n) $.
  3. It uses $ O (n ^ {k / 2}) $ Memory.

There are several ways to deal with the first difficulty. We could just repeat the whole algorithm $ sqrt {k} $ times. Alternatively, there could be deterministic ways to achieve the same goal. Here is a hybrid solution:

  • Random partitioning $ A $ in $ k ^ 3 $ Parts $ P_1, ldots, P_ {k ^ 3} $ (everything much larger than $ k ^ 2 $ would work). It is highly likely that each element is the optimal solution in itself.
  • Consider all possible partitions from $ A $ in two parts of the form $ P_1, ldots, P_i $ and $ P_ {i + 1}, ldots, P_ {k ^ 3} $. One of them is included exactly $ k / $ 2 Elements of the optimal solution.

To deal with the second difficulty, we need to be a bit more careful when implementing it. With the merge (mergesort's well-known subroutine) it should be possible to calculate this $ k / $ 2-Sum of each part in $ O (n ^ {k / 2}) $. The last step can be implemented in $ O (n ^ {k / 2}) $ using a classic two-pointer technique (the first pointer goes up in the first half, the second goes down in the second half).

There are tricks to reduce memory usage $ O (n ^ {k / 2}) $ to $ O (n ^ {k / 4}) $: Divide each part further into two parts. You can easily go through everything $ k / $ 2-Sums in the first part taking into account all pairs of $ k / $ 4-sum in its subsections. The binary search in the second part can then be implemented using the two-pointer technique.

Split a large list to avoid the threshold for viewing in Sharepoint Online

Managing a large list in SharePoint has always been a challenge because personal data analysis of the list is very important. Therefore, we can split the list items into multiple views based on the data analysis and behavior of the columns, rather than managing multiple lists the some condition.

For example, if you have a selection column with the values ​​"Yes" and "No". We can create two views as follows:

View 1

  • With the condition "Yes" and columns created or modified

View 2

With the condition "No" and columns created or modified

The division even with more views depends on the different metadata and the frequency of the updates.

The view filter logic should look like this:

To view 1:

Column 1 eq "Yes" and created> = (today) – 3

For view 2:

Column 1 eq "No" and created> = (today) -3

Remarks :

The filter above shows data that has been created with yes and no values ​​for the last 3 days. This is just for example, you can change it according to your requirements.

And so on …

The bottom line of the approach above is that the view contains fewer elements at a particular point in time (less than 5,000).

For implementation details, see the following article:

List view threshold error (5000 item problem) in SharePoint On-Premise / SharePoint Online

Remarks::

  • It is recommended to implement an archiving policy in the list with which the old elements are to be moved to the archiving list regularly.

machine learning – neuron: question about NOT gate and threshold

This is a neuron of a NOT gate, as shown in Rojas' ML book. I have a question about behavior. As I understand it, the neuron generates a signal when it is entered $ x_1 $ are summed up $ geq 0 $,

In the picture below, if $ x_1 $ if 0, then threshold 0 is reached and there is an output from this neuron. If $ x_1 $ is 1, has the threshold been reached and the neuron outputs 1?

Estimate whether someone can give guidance on how this neuron works with the inhibitory signal it contains.

Thank you very much

Not

Distributed Systems – Why is a Byzantine Fault Tolerance Threshold Set on an "Asynchronous" Network? (where it can't even tolerate a faulty node)

In the following answer (SHORTCUT: https://bitcoin.stackexchange.com/a/58908/41513) has been shown for Asynchronous Byzantine Agreement::

"We cannot tolerate that 1/3 or more of the knots are dishonest or we lose
either security or liveliness. "

The following conditions / requirements were taken into account for this verification:

  1. Our system is asynchronous,
  2. Some participants may be malicious.
  3. We want security.
  4. We want liveliness.

A basic question is:

Taking into account the well-known paper with the title: "Impossibility of a distributed consensus with a faulty process" (SHORTCUT: https://apps.dtic.mil/dtic/tr/fulltext/u2/a132503.pdf)

Shows that:

No fully asynchronous consensus protocol can tolerate a single unannounced process death.

Can we still assume that the network is asynchronous? As in this case, the network cannot even tolerate a faulty node.

co.combinatorics – Reference – Lower limit of probability threshold for Ramsey properties

I look at a paper by Rodl and Rucimski: Threshold function for Ramsey properties, also available semantically scholarly. There they indicate the following:

For all integers $ r geq 2 $and for each chart $ G $ What is not a star forest is constant $ c $ and $ C $ so that
$$ lim_ {n to infty} mathbb {P} (K (n, p) to (G) _r ^ 2) = left {
begin {array} {ll}
0 & text {if} pCn ^ {- 1 / m_G ^ {(2)}}
end {array} right. $$

Where $ K (n, p) $ is the usual binomial random graph model, and $ H to (G) _r ^ 2 $ the usual Ramsey notation, which means that none $ r $– coloring the edges of $ H $ contains a monochromatic $ G $,

They prove the 1 statement in the same paper. The 0 instruction is written

The proof of the 0 statement from Theorem 1 & # 39; appeared in (RR93)

And the reference is

(RR93) _, Lower bounds for probability thresholds for Ramsey properties, Combinatorics, Paul Erdos is eighty (Volume 1) Keszthely (Hungary), Bolyai Soc. Mathematics. Studies, 1993, pp. 317-346

I found only one source for this paper, Rucinski's own website.

There, however, the 0 statement is not proven for any number of colors, only for 2 (or I missed the generalization). The proved sentence is

For every diagram $ G $ What is not a star forest, there is a positive constant $ c_G $ so that
$$ lim_ {n to infty} mathbb {P} (K (n, cn ^ {- 1 / m_G ^ {(2)}}) to (G) _2 ^ 2) = 0 $$

The paper contains only one final note

Added as proof. Recently the authors have proven sentence 3 for any number of colors.

I found a dozen papers related to that full result (I.e., $ r $ Colors), all with the same references, but not the actual paper.

Can someone point me to a place where I could find him?

To edit: I should add that I was unable to access the originally quoted magazine "Paul Erdos is Eighty Vol1" online or in print.

List View Threshold – Count the number of documents in an LVWP on a WP page WITHOUT summation

I have a Web Part for the List View of the Document Library (LVWP) on my SPO page.

Because of the threshold of 5,000 items, I had to turn off all counts in the Totals menu in the List View Editor. However, I would like to know how many documents / elements are displayed in the LVWP.

Is there any code or script that can be inserted into a wp script editor to accomplish this?

Threshold warning for list / library view with filter for indexed column

After the following elements:

  • The library view threshold is 39000 items (configured at the farm level).
  • The library contains 50,000 items
  • We created a view with a filter for an indexed column that matches only one element
  • Search works well on the site collection page

However, I still have the warning even though this view is in use: the view cannot be displayed because it exceeds the threshold for the list view.
Do you have any idea where that could come from?

Kind regards,
Bastien