Cake Development Corporation

Collective intelligence is a shared or group intelligence that emerges from the collaboration and competition of many individual. Some examples would be Amazon's book recommendations. Netflix uses collective intelligence to track their stock and supply changes, and more accurately allocate their resources.

Recommendations requires something to track like sales, or user preferences. A group of users is also required, the larger the group the better. A research survey found that a random sample of less that 1500 people will be the ideal and most efficient sample size.

There are many ways to estimate preferences. The most simple and basic one is Euclidean distance scoring. Pearson Correlation Score, also uses 2 axis to plot out the points, this method is used by amazon and delicious.

Stochastic Optimization

Theory of being able to optimize things. Requires a cost function, it is the most important part of optimization as it gathers the dataset used. Ie. Execution time of a request, time used for data integrity, error handling/logging, client bandwidth, data queries/filtering. When doing metrics and calculating cost you can use a unit cost and parts to unit. This will allow you to control and manage the final output. Depending on what you are measuring, you need to control different weighting. Random searching is an inefficient process, as it is difficult to determine whether you are moving toward or away from where you want to be. Writing unit costing functions is a specific to the task you want to achieve. When doing optimization with cost optimization, you need to be aware of local minimums. Once you have reached a new minimum or optimized level you need to do additional testing by expanding the range in either direction. By expanding your test range you help to ensure that you have found a more accurate low cost point, and aim to ensure that you haven't found a local low point.

Genetic Algorithms, and Hierarchal Grouping are other statistical methods to look into. Ryan is going to be releasing both Pearson Correlation and Euclid distance scoring will be released on the The bakery and SerenitySoft So check back for some code.