I am not sure where I heard it, but all great things must begin humble… I will work on this “opus” in three sections: The first will be what I got out of class, the second what I learned through coding, and the third what I got out of my honors level reading and how the material I learned in class and lab relate to the material taught in both (and also just things I found interesting).

The journey begins

Well not so much… the journey began around 08/24/2015, but this is a summary of what I have learned so far.

Scope: I can understand how this could be very useful for controlling the flow of information contained in variables.

Functions: input in, output out. It gets much, much, more complicated, but a little abstraction can help manage that.

Parameters are the items contained in the parenthesis. It is vital to make sure the parameters are of the right type and order as is required by the specific function.

Data types made me want to pull my hair out. Perhaps it was just java… or maybe c makes this even more difficult. So far, my money is on java is just… annoying (as much as I love it)

If one uses an array, and tells it something like int c[2], it sets aside twice the memory.

A struct is like an array but can have differing data types.

Types of output functions: sprintf, printf (apparently just automatically puts the stdout for the file), fprintf.

Any number preceded by 0x is hexidecimal or base 16.

a preceding 0 is base 8, or octal.

You can also use format specifiers (for example fprintf(stdout, “a is: %d” (this one caught me, you need to use one format specifer per variable)

We created a chart of the first sixteen digits of Base 2, Base 8, Unsigned Base 10, Signed Base 10, and Base 16.

We delved in much greater detail into pointers and arrays.

We discussed the malloc function as well as the sizeof command(?)(I know it isn't a function…)

The concept of coding dynamically came up… this is something I will have to keep in mind.

incrementing prior to a variable and after a variable was shown and used in a variety of programs.

relational operators were discussed, something I could have made great use of in problem solving.

the importance of using fewer if statements and nested programming structures was shown to be imperative

fetch, check, execute, and store…the four points of an instruction.

and finally an interesting example of the use of arrays was utilized in class to make an alternate “Hello World” program.

We discussed in some depth the uses of loops and containers during this two week period. We wrote a variety of different programs to demonstrate their usefulness. We wrapped up studying the third container type (the union) and why it isn't used a whole lot. I found most of this very useful but became a bit frustrated with the unions as it was giving me some errors.

We discussed functions, primarily how to create them and call them. We also looked at how structs and arrays are utilized from within functions.

We also looked at the gd graphics library, how to manipulate files, and the uses of command line arguments

We covered many topics in C++ including public, private, inheritance, classes etc.

We covered .o and .h files and how to include them in lab. We also looked at link lists.

We worked with digit.cc (creation of a base three number)

I have had a considerable amount of success programming over the last two weeks. I finished both class projects and even successfully finished a new program of my own “triples”, which as you can guess computes the square root of any three digit number that ends in five. I may try to figure out a way to get it to do the others, but I need to get my honors work done.

I already had my projects done for the week and alas another class demanded far more of my time than I would have appreciated :(

I had a great deal of fun writing the multby11 program! It was very frustrating at times but I feel my logical reasoning skills have improved due to this project. The testing of this program probably took far longer than the writing of it, but I am extremely confident that it operates as expected.

We programmed a version of the multby11 program utilizing arrays and loops. For whatever reason my array wanted to print backwards, and I had some difficulty getting a simple modulus to work for me. Nonetheless I had fun working on this project and worked on it far longer than I probably should have.

I completed a new program- Circle of Squares. Despite being a bit overwhelmed, I still found this project to be fun and plan on revisiting it in the future.

I am now working on a new project, “All From Nine”, the basic concept of taking the input, loading an array, doing the arithmetic, and printing all fairly simple. The complexity arises with the use of functions and pointers which I haven't used before.

Completed my all for nine program. Had some difficulty with my printarray function but the other ones worked just fine. Output was as expected when I removed that function.

I have my command line arguments and menu selection logic complete for sam0. I need to do the actual encryption and decryption programming, but after that it should be functionally complete. (10/30/2015)

Much of the encryption programming has been completed, but I need to finish that and the decryption end of things and add the calls. Then on to testing and debugging. (11/02/2015)

I found a number of errors that bogged my efforts down. Some work is still needed on sam0.

I managed to get the encryption side of this done but ran out of time to complete the decrypt side.

Worked on some nifty binary manipulation to decrypt an image. Works nicely, but is not perfect

Some of the background and history of the field including those approaches limitations

“Artificial Neural Networks are information processing systems whose structure and functionality simulates the nervous systems and particularly the brain of animals and human beings.” They function (paraphrasing) by collecting a group of neurons (each an individual processing unit) with which information passes through them layer by layer, each neuron both taking and recieving information. These all run simultaneously with each other.

Evolutionary Algorithms is a collected group of solutions that are given a number of attributes that calculate to a given fitness score depending upon their success in the constructed environment. These solutions are then pared down according to their fitness, and mutations of the original successful solutions are introduced to compete against them. (←- this is actually memory from a program that I used to play around with that used these to construct artificial life forms)

In Fuzzy Systems, “vague knowledge is formalized by the help of fuzzy logic and fuzzy set theory.” These are good for getting approximate or “close enough” answers when deriving the true answer may be cost prohibitive. (paraphrase)

“Bayesian Networks are means to efficently store and reason with uncertain knowledge in complex application areas.” This utilizes “probabilistic graphical models”, that allows the user to “draw reasonable inferences on new information.”

History and background of artificial neural networks including what they can and cannot do. (or rather what they do well and what they do not at least yet).

Biological basis of the artificial neural network; 100 billion neurons in the human brain. These all act in parallel. “Neurons process information primarily by interacting with one another.” (Note to self; need to study the brain in more detail)

The text utilizes “threshold logic units” to mimic the neuron. The most common name is “Perceptron”, although they “are somewhat more complex.” “If a neuron receives a strong enough excitory input that is not compensated for but an equally strong inhibitory input it becomes active and sends the signal to the next level.”

Geometric interpretation of the decision to output a 0 or a 1. Talks about decision borders, separating lines and planes (I admit I only vaguely understood the math here although on an intuitive level it made sense).

Discusses the limitations to the method given in the preceding section. Only works with those functions which are linearly separable. An example of a function that is not linearly separable is a biimplication, or x1 ←→ x2. Presents proof of this.

Considers how the powers of a threshold logic unit scales greatly if one combines them into networks (paraphrase). Presents math considering hyperplanes and hypercubes.

Training the parameters. The geometrical advantage is that it separates the plane into two sections: that of 0, and that of 1 (paraphrase; given the excitory input has to outweigh the inhibitory input, I can see the value of this). This section also discusses errors and some methods to correct them, online learning, and batch learning. Presents algorithm 3.2 which deals with the online training of a threshold logic unit, and algorithm 3.3 which deals with the batch training of a threshold logic unit.

variants; delta rule, Widrow-Hoff procedure, error correction procedure.

   Discussion of error back propagation.
   

Structure of Neural Networks. Vertices, edges, nodes. “connections between neurons are always direted” vertex u –> vertex v, predecessors/successors. the line between nodes (or neurons) is called a connection. meurons are divided between input neurons, output neurons, and hidden neurons. each neuron possesses the following functions: network input function, activation function, and output function.

Distinguish two separate forms of neural networks: feed forward networks that contain no loops or directed cycles and recurrent networks that contain one or the other. (all mostly paraphrasing)

Operation of a neural network; Divide the computation of the network into two phases: the input phase and the work phase. The input phase is when all “the external inputs are fed into the network”, and the work phase, “is when the output is computed.”

• a) input phase initialize network, activations of neurons are set to values of corresponding external input.
• b) In the work phase the external inputs are switched off. Activations and outputs are recomputed.

a & b may happen multiple times. This happens until network reaches a stable state or further recomputations do not change the output.

Training a neural network; discusses fixed learning tasks, training patterns, input and output vectors.

Utilizes a strictly layers structure. introduces the concepts of bias value, unipolar and bipolar sigmoid functions. Fredkin gates; plays an important role in so called conservative logic. Three inputs, s, x1, and x2 as well as three outputs s, y1, and y2. The inputs x1 and x2 are connected either parallel or crossed to the two outputs y1 and y2.

Function Approximation; discusses the step by step functioning of the network.

Logistic Regression; method of least squares, also known as regression. (sigh); best fit functions, logistic functions, linearizing the function. Utilizing “logit transformations.” multi-linear regression.

gradient descent. differentiable activation function. vector field, partial and direction derivatives. computation of the learning rate.

Error back propagation is discussed. The general definition of this is, “an error signal is transmitted from the outer layer backwards through the hidden layers (of the neural network).” This allows error to be detected in the hidden layers (and corrected).

The gradient decent method of error backpropagation is shown to greatly reduce errors in online training and batch training. Error does not completely disappear, but it is greatly reduced.

Variants of the gradient decent method of error backpropagation are discussed. The importance of choosing differing initial values is emphasized to avoid falling into a local minimum but it is admitted that this is not entirely possible.

(The variants are:

Manhattan Training: modifies the weight adaption rule so that the dependence on magnitude is avoided.

Lifting the derivative of the activation function: modification of the gradient decent is also known as flat spot elimination. This speeds up training in those instances where the gradient is very small or is gone entirely.

Momentum Turn: “Introducing a momentum turn can speed training in the parameter space in which the error function is fairly flat but has a uniform slope in one direction”

Self Adaptive Error Backpropagation: “Learning rates are not constant but adapted prior to their use depending upon the current and previous gradient”

Resilient Error Backpropagation: combines the ideas of Manhattan training and self adaptive error back propagation.

Quick Propagation: calculates the error function at the current location of the current weights by a parabola.

Weight Decay: The small decay of the weights of each neuron so that they do not become excessively large.)

Gives specific examples of the previous section.

Discusses sensitivity analysis. This determines “what influence the different inputs have on the output of the network.”

This is done “by summing the derivatives of the output w.r.t, the external inputs over all output neurons, and all training patterns.”

Radial Basis Function Networks are “feed-forward neural networks with a strictly layered structure. They possess exactly three layers.”

Also discusses radial and distance functions.

Function Approximation: Details and graphs many interesting functions. I am starting to understand some of these squiggles…

x–> x1, x2, x3, x4. x1: to delta, then to y1, and finally to output y x2: to delta2, then to y2, and finally to output y x3: to delta3, then to y3, and finally to output y x4: to delta4, then to y4, and finally to output y

(from figure 6.7)

Initializing the parameters. Treated in passing… choose random numbers of whom the absolute value is less than or equal to .5.

Fascinating discussion of matrices.

Training the parameters: Also trained by gradient decent.

Considerable discussion of adaptation weights and how they differ from the previous network.

Math I can understand! d(x,y)=sqrt[(x-y)^T - Series^-1 *(x-y) is the Mahalanobis distance which can describe ellipses with arbitrary orientation. This gives a very interesting solution space X2

New Neural Network: Self Organizing Maps, also known as Kohonen feature maps. These are effectively radial basis function networks without an output layer, or rather the hidden layer functions as the output layer.

Learning vector quantization- organizes the data into clusters

Utilizes competitive learning amongst the data points. “Each one is traversed in turn one by one. with each set of data a competition is carried out which is won by the output neuron that yields the highest activation energy”

Discusses the differences between online and batch training for this particular type of network.

Neighborhood of the output neurons; need to pay attention to the neighborhood relationship otherwise it isn't a true self organizing map.

Hopfield Networks; the preceding chapters dealt with so called feed forward networks… this and the next chapter deal with what is referred to as recurrent networks. These are networks that contain directed cycles. The Hopfield network is an example of one of the simplest of these. These contain no loops (ie the input is never the output). Activations occur with synchronicity (or at the same time).

Convergence of computations: It appears that updating with synchronously isn't as good as doing so asynchronously due to a stable state being reached is always achieved in the latter example.

The goal here is to apparently converge the computations to a locally minimal state (and to do so, it is vital no neurons are left out… each needs to have an activation update

Associative memory: the kind of memory addressed by its contents. “Pattern presented to associative memory–>the memory returns whether or not the pattern coincides with the with one of the stored patterns. The memory may also return a stored pattern that is similar to the one presented.” Hopfield networks may be utilized as associative memory.

this is taking me longer because I am considering implementations of C code to actually do some of the algorithms presented

The use of the Hopfield network to solve optimization problems such as the traveling salesman problem.

Application of simulated annealing to Hopfield networks.

various new examples of neural networks utilizing differential equations (which mostly went over my head).

more differential equations; the diagram on page 150 makes the recurrent neural network make a little more sense but the math is still mostly over my head.

vectorial neural networks and their various uses, for example calculating oblique throws, or the orbits of planets.

error back propagation in time.

Equations for straight lines

Regression

Active transformation

“Metaheuristics are usually applied to problems for which no efficient solution algorithms are known”

Discusses things such as guided random searches and anytime algorithms.

Biological Evolution

“Beneficial traits resulting from random variation are favored by natural selection.”

Discusses differential reproduction… Diversity, Variation, Inheritance, Speciation, Birth Surplus/Overpopulation.

Simulated Evolution.

Optimization Problems. Problems and solutions.

Basic notions and concepts

Building blocks of an evolutionary algorithm. (encoding, initial population, fitness functions, selection method, genetic operators, termination criterion, parameters.

The N Queens Problem.

page 181-185 (potential program to work on)

Related Optimization Techniques

Gradient Ascent or Descent

Hill Climbing

Simulated Annealing

Threshold Accepting

Great Deluge Algorithm.

Record to Record Travel

The traveling salesman problem (utilizing evolutionary algorithms instead of neural networks)

Encoding of Candidate Solutions. Desirable properties for encoding: 1) Similar Phenotypes should be represented by similar genotypes, 2) Similarly encoded candidate solutions should have similar fitness, and 3) If possible, the search space should be closed under the genetic operators.

Hamming Cliffs. (a term denoting the difficulty in zeroing in on a solution improvement process due to the number of bits needing to undergo mutation and crossover)

Epistasis. “In biology epistasis means the phenomenon that one allele of a gene(the so called epistasis gene) suppresses the effect of all possible alleles of another gene. It is even possible for one such gene to suppress multiple other genes.

More discussion on the traveling salesmen problem.

Closeness of the search space. This section describes potential solutions that lie outside of the search space and solutions to that problem.

Discusses Fitness and Selection.

Fitness Proportionate Selection. Roulette Wheel Selection.

The Dominance Problem. Discusses the potential of an individual with very high fitness dominating all others (thus suppressing them). Crowding and Premature convergence are also discussed.

Vanishing Selective Pressure. Discusses the possibility of no candidates having a high enough fitness score to push them forward through the selection process.

Adapting the Fitness Function. Scaling the fitness function through the use of linear dynamic scaling. Discusses an alternate time-dependent fitness function called Boltzmann selection.

The Variance Problem. What it is and how to solve it.

Rank-Based Selection. Ranking a population based upon fitness scores. (Incidentally the method I used when playing around with an application that utilized evolutionary algorithms without knowing what it was called… it simply seemed intuitive).

Tournament Selection. Individuals from the population are drawn at random and pitted against one another. The one with the highest fitness wins. Ties are broken arbitrarily. Various other selection methods are summarized.

Genetic Operators. What is applied to a certain number of members of the population every generation.

Mutation Operators. Includes some interesting algorithms.

Crossover Operators. A cut is chosen at random, and gene sequences are exchanged. Can be single, double, or n-point crossovers. Also discusses uniform, shuffle, and uniform-order crossover. Edge recombination is a crossover designed specifically for the traveling salesmen problem.

Multi-Parent Operators. Discusses very briefly the idea of diagonal crossover.

Characteristics of Recombination Operators. Details positional and distributional bias.

Interpolating and Extrapolating Recombination. Presents arithmetic crossover. Also discusses another item that I ran into with my experimentation, the so called Jenkins nightmare, in which your population lacks sufficient variation and any crossover only results in more similar children.

Genetic Algorithms. Algorithm 13.1 provides a skeleton. I will make an attempt to translate this to C.

The schema theorem. Details a lot of math that is currently over my head regarding fitness selection.

The Two-Armed Bandit Argument

The Principle of Minimal Alphabets. The claim that certain binary encodings are optimal in a certain sense.

Evolutionary Strategies. These are the oldest form of evolutionary algorithm. They ignore crossover in favor of mutation.

Selection. Details the use of the elite principle when dealing with evolutionary strategies. Also discusses the plus and comma strategies.

Global Variance Adaptation. Presents an algorithm.

Local Variance Adaptation. Another algorithm. Summarizes the the standard deviation array of chromosome-specific (and gene specific) mutation step sizes.

Covariances. covariance matrix, Eigen decomposition. Covariation matrix evolution strategy.

Recombination Operators

Genetic Programming. Programs that associate a certain set of inputs to a certain set of outputs to solve a specific problem.

Initialization; creation of the initial population (algorithm 13.5, 13.6, 13.7)

Genetic operators. Discusses crossover and mutation. Presents figures 13.9 and 13.10 to show the concept graphically.

Application examples

The 11-Multiplexer Problem deals with 8 data values and 3 address lines. Figure 13.12 shows the idea in pictorial form.

Symbolic regression with constants. Introduces ephemeral random constants as a method to solve the difficulties that arise from this problem.

The Problem of Introns; describes what an intron is and gives three potential solutions (editing, brood recombination, and intelligent recombination)to the problem of excessive intron buildup.

Extensions. Various improvements and suggestions discussed that others have mentioned including encapsulation. Other extensions include iterations and recursion.

Other population based approaches. Swarm intelligence, Self organization, Population based incremental learning.

Population based incremental learning algorithm. Introduces the Bayesian optimization algorithm.

Ant Colony Optimization. This is a rather extensive treatment of this idea. Presents multiple algorithms including a link to one that solves the traveling salesmen problem.

Particle swarm optimization.

Behavioural Simulation. Utilization of game theory. Very useful in genetic algorithms.

The prisoners dilemma. Discusses the pay off matrix, and describes the Nash Equilibrium as, “a situation in which neither agent can improve its pay off by changing its choice without the other also changing their choice”

The iterated prisoners dilemma.

A genetic algorithm approach.

Extensions to the setup.

Multi-criteria Optimization.

Weight Combination of Criteria. Discusses the problem of aggregate preferences. This sometimes results in something called Arrows paradox, which can be avoided through something called scaled preference assignments.

Pareto-Optimal Solutions. ergo what happens when any one criterion harms another.

Finding Pareto-Frontiers with Evolutionary Algorithms An alternative is the Vector Evaluated Genetic Algorithm. Power law sharing is utilizing a similarity measure for individuals of the population and not the structure of the chromosome. Presents another alternative called the non dominance sorting algorithm.

Parallelization

Parallelizable Operations

Island Model and Migration

Cellular Evolutionary Algorithms

Combination with Local Search Methods

“The main aim of fuzzy logic and fuzzy sets is to overcome the disadvantages of classical logic and classical sets”

Natural Languages and Format Models

When a problem can be specified in concrete terms conventional mathematics are generally adequate to solve a problem. In some instances this is not the case however, such as in specifying an action in linguistic terminology.In that instance a different technique is necessary.

Fuzziness versus Uncertainty

Something is “said to be fuzzy when its meaning is not fixed by sharp boundaries.” “The gradual degrees of this membership is also called fuzziness.”

The Sorites Paradox is when a statement such as “a heap of sand is small, adding one grain to it and the heap is still small,” therefore all heaps of sand are small… the problem being that when taken together a beach is obviously not small. But this paradox dissolves if the degrees of “heaps of sand” slowly decreases by adding one grain after another. The propositions “truthiness” if you will decreases proportionally.

Also discusses the concept of uncertainty.

Fuzzy Sets

”…fuzzy sets can be considered as generalized characteristic functions“

“a fuzzy set is formally defined as a set of pairs, each consisting of one argument of the function and the image of this argument”

Interpretation of Fuzzy Sets

If 0 is interpreted as lack of membership, and 1 is considered to be complete membership, what is the meaning of .6, or .8? It is important to have a consistent interpretation of fuzzy sets so as not to introduce inconsistencies.

This section also denotes the difference between probability and membership.

Representation of Fuzzy Sets

Definition based on Functions

triangular and trapezoidal functions

Level Sets

discusses the relationship of level sets to their fuzzy sets

Fuzzy Logic

There are three meanings to the term: the first is the general idea of fuzzy logic, the second deals with logic as it is applied to things like expert systems, and the third considers fuzzy logic from the point of view of multi valued logic and is devoted to issues connected to logical calculi and the associated deduction mechanisms.

Propositions and Truth Values

Discusses basic logic and truth values (review for me as I have already had a formal logic class)

t-Norms and t-Conorms

truth functionality; “truth value of the combination of several propositions depends only on the truth values of the propositions but not the individual propositions.”

Intersection; “the underlying concept of generalizing set- theoretic operations to fuzzy sts is explained in detail for the intersection of fuzzy sets”

Describes the union of fuzzy sets

Discusses the Complement of fuzzy sets

Linguistic modifiers and how they modify the membership degree

Mappings of Fuzzy Sets

discusses the extension principle, which holds that the membership degree of an element(y) to a specific image(Mu of X) under a specific mapping(X → Y) is the greatest possible membership degree of all x to Mu that are mapped from y under f.

Mapping of level sets

Another way to characterize the image of a fuzzy set is to determine its level sets.

Cartesian Product and Cylindrical Extension

utilization of the extension principle for fuzzy sets with more than two sets

a cylindrical extension is a special case of the Cartesian product

extension principle for multivariate mappings

“relations can be used to model dependencies, or connections between variables, quantities, or attributes.”

Crisp Relations

Application of Relations and Deduction

Chains of Deduction

“inferring new facts from rules and known facts usually means that we deal with chained deduction steps”

Simple fuzzy relations

composition of fuzzy relations

similarity relations can be used to characterize the inherent ambiguity or vagueness of fuzzy systems

Similarity relations are fuzzy systems that specify for pairs of elements or objects how indistinguishable they are

Fuzzy Sets and Extensional Hulls

the property of extensionality is characterized by the idea that similar members of a set have similar membership degrees… i.e. elements that are nearly indistinguishable should have behave similarly or have similar properties

Scaling Concepts

discussion of how the membership degree should not depend upon the measuring units (i.e. height should have a similarity degree that is the same regardless of whether a person is measured in metric or standard)

Additional interpretation of fuzzy sets

Possibility Theory is an alternative form of rigorous interpretation of fuzzy systems. It is especially useful for exploring discrete universes of discourse.

“the biggest success of fuzzy systems has been fuzzy controls”

Mamdani Controllers

A Mamdani Controller is based on a finite set of if-then rules… “although rules are formulated in terms of an if-then should not be interpreted as logical implications but instead as a piecewise defined function”

Takagi-Sugeno-Kang Controllers

These are similar to Mambdani controllers except the consequent of a single rule is no longer a fuzzy set but determines a function with the input variables as arguments

Logic Based Controllers

Mamdani controllers and similarity relations

interpretation and construction of a controller

Hybrid Systems to Tune Fuzzy Controllers

discusses the combination of other forms of computational intelligence and fuzzy systems

The oldest fuzzy approach to data analysis

Fuzzy Methods in Data Analysis

fuzzy data analysis can be interpreted in two distinct ways: the first is analyze the data by fuzzy sets and then analyze the fuzzy data. This leads to two views: in the epistemic view fuzzy sets are used to represent incomplete knowledge about an underlying object or precise quantity, whereas the ontic view fuzzy sets are considered as real complex humped entities. The second interpretation is fuzzy techniques for the analysis of (Crisp) Data… it states that although data can be noisy or imprecise, it remains crisp in this approach and fuzzy techniques are applied to analyze the data.

Clustering

the general objective of clustering is to group objects in such a way that objects in the same cluster are as similar as possible. A common approach to describe the clusters is to use prototypes that capture the location and possibly even the shape and size of the clusters in the data space.

Presuppositions and notation

Classical e-Means Clustering is used to find the minimum since analytical means oftentimes fail.

Fuzzification by membership transformation (modifies the membership degrees… lots of math here)

Fuzzification by membership regularization is used when one does not want to alter the membership degrees. Instead a regularization term is added to the objective function.

comparisons… entropy regularization seems to have equivalent results without the drawbacks of normal quantum regularization.

relational database systems, databases, database management, and database theory

A Fictious Example; discusses the example of an automaker that keeps databases for it suppliers. Drawing inferences from the tables are presented, modeled, and exploited.

Elements of probability and graph theory

Probability Theory

Presents Kolmogorov axioms. “An event in this nomenclature is simply a set of elementary events that are distinguishable.(i.e. have an identity)”

Random Variables and Random Vectors

Independencies

Graph Theory… presents needed knowledge of directed, acyclic graphs and undirected graphs to handle Bayes and Markov networks.

Decompositions

connects the concepts of conditional independence with separation in graphs

Dependence Graphs and Independence Graphs

Dependence graphs represent all conditional independences that are valid in the distribution of p but may contain additional ones that are not valid in p.

An independence graphs on the other hand encodes only those conditional independencies that are valid in the distribution of p.

A Real World Application

Evidence Propagation

Initialization of a Bayes network

Message Passing

after initialization, member cliques exchange messages to inform each other of (a possible) change in their potential functions.

Update

After all messages have been sent each member decides whether or not to change

Marginalization is minimized by looking at each attribute A for the smallest clique C that it contains

Derivation

derives the essential equations necessary for evidence propagation

Other propagation algorithms

Learning Graphical Models