MACHINE LEARNING
Kennedy B. Mateo
4BSCSA
Francis Wednesday Samonte
LEARNING Data and Goals >representation of heuristics as
through the course of Type of data problem-solving rules
their interactions with – positive or negative A Set of operations
the world examples >Given a set of training instances,
through the experience – Single positive the leaner must construct a
of their own internal example and domain generalization, heuristic rule, or
states and processes specific knowledge plan that satisfies its goal
Is important for practical – high-level advice (e.g. >Requires ability to manipulate
applications of AI condition of loop representations
termination) >Typical operations include
Definition: – analogies(e.g. – generalizing or
Any change in a system that electricity vs. water) specializing symbolic
allow it to perform better the Goal of learning algorithms: expressions
second time on repetition of the acquisition of – adjusting the weights in
same task or on another task – concept, general a neural network
drawn from the same population description of a class of – modifying the program’s
(Simon, 1983) objects representations
– plans
MACHINE LEARNING: – problem-solving Concept space
AI systems grow from a minimal heuristics >defines a space of potential
amount of knowledge by learning – other forms of concept definitions
Categories: procedural knowledge >complexity of potential concept
Symbol-based learning Properties and quality of data space is a measure of difficulty of
Inductive learning -- – come from the outside learning algorithms
learning by examples environment (e.g.
Supervised teacher) Heuristic Search
learning/unsupervised – or generated by the >Use available training data and
learning program itself heuristics to search efficiently
Concept learning –- classification – reliable or contain >Patrick Winston’s work on
Concept formation -- clustering noise learning concepts from positive
– Explanation- – well-structured or and negative examples along
based learning unorganized with near misses
– Reinforcement – positive and negative >The program learns by refining
learning or only positive candidate description of the
Neural/connectionist target concept through
networks Representation of learned generalization and specialization.
Genetic/evolutionary learning knowledge – Generalization changes
>concept expressions in the candidate
I.A Framework for Symbol- predicate calculus description to let it
Based Learning – A simple formulation accommodate new
Data and goals of the of the concept learning positive examples
learning task problem as conjunctive – Specialization changes
Representation sentences containing the candidate
Language variables description to exclude
A set of operations >structured representation such near misses
Concept space as frames >Performance of learning
Heuristic Search >description of plans as a algorithm is highly sensitive to
Acquired knowledge sequence of operations or the quality and order of the
triangle table training examples
that covers the positive
Notion of covering examples, c c’
>>If concept P is more general
than concept Q, we say that Maximally general specialization
“P covers Q” A concept c, is maximally
>>Color(X,Y) covers color(ball,Y), general if it covers none of the
which in turn covers negative training instances, and
color(ball,red) for any other concept c’, that
Concept space covers no negative training
II. Version Space Search >>Defines a space of potential instance, c c’.
Implementation of concept definitions
inductive learning as >>The example concept space Combining the two directions
search through a concept representing the of search into a single
space predicate obj(Sizes, Color, algorithm has several benefits.
Generalization Shapes) with properties and G and S sets summarizes
operations impose an values the information in the
ordering on the concepts Sizes = {large, small} negative and positive
in a space, and uses this Colors = {red, white, blue} training instances.
ordering to guide the Shapes = {ball, brick, cube}
search An incremental nature of
9.2.1 Generalization The candidate elimination learning algorithm
Operators and Concept algorithm Accepts training
Space instances one at a time,
9.2.2 Candidate Version space: the set of all forming a usable,
Elimination Algorithm concept descriptions consistent although possibly
with the training examples. incomplete,
Generalization Operators and >Toward reducing the size of generalization after each
the Concept Spaces the version space as more example (unlike the
examples become available batch algorithm such as
*Primary generalization – Specific to general ID3).
operations used in ML: search from positive
examples Even before the algorithm
>Replacing constants with – General to specific converges on a single concept,
variables search from negative the G and S sets provide usable
color(ball, red) -> color(X, examples constraints on that concept
red) – Candidate elimination – If c is the goal concept,
> Dropping conditions from a algorithm combines then for all g∈G and s∈S,
conjunctive expression these into a bi- s≤c≤g.
shape(X, round) ^ size(X, directional search – Any concept that is more
small) ^ color(X, red) >Generalize based on general than some
-> shape(X, round) ^ color(X, regularities found in the concept in G will cover
red) training data negative instance; any
>Adding a disjunct to an >Supervised learning concept that is more
expression specific than some
shape(X, round) ^ size(X, The learned concept must be concept in S will fail to
small) ^ color(X, red) general enough to cover all cover some positive
-> shape(X, round) ^ size(X, positive examples, also must instances
small) ^ (color(X, red) be specific enough to exclude
color(X, blue)) all negative examples LEX: Inducing Search
> Replacing a property with its Heuristics
parent in a class hierarchy maximally specific generalization
color(X, red) A concept c, is maximally Lex learns heuristics for
-> color(X, primary_color) if specific if it covers all positive solving symbolic
primary_color is superclass of examples, none of the negative integration problems.
red examples, and for any concept c’,
Goal: searching for the classifications 3. Refine the hypothesis by
expression to be integrated that (if n=50, larger than adding a new rule,
contains no integral sing. the number of whose premises are the above
Architecture of LEX: molecules in the conditions, and
1.Generalizer- that uses universe) whose consequent asserts the
candidate elimination to find – We need additional target concept
heuristics heuristics META-DENDRAL
2.Problem solver- that produces (assumptions) to >Learns rules for DENDRAL
traces of problem solutions restrict the search > Remember that DENDRAL
3.Critic that produces positive space infers structure of organic
and negative instances from a Inductive bias refers to the molecules from their chemical
problem trace assumptions that a formula and mass spectrographic
4. Problem generator that machine learning data.
produces new candidate algorithm will use during >Meta-DENDRAL constructs an
problems the learning process explanation of the site of a
– One kind of inductive cleavage using
Evaluating Candidate bias is Occams Razor: >>structure of a known
Elimination assume that the compound
The candidate simplest consistent >>mass and relative abundance
elimination algorithm hypothesis about the of the fragments produced by
demonstrates the way in which target function is spectrography
knowledge representation and actually the best >>a “half-order” theory (e.g.,
state space search can be applied – Another kind is double and triple bonds do not
to the problem of machine syntactic bias: assume break; only fragments larger than
learning. a pattern defines the two carbon atoms show up in the
Search Based learning class of all matching data)
Like all search problems, strings These explanations are used as
must deal with the o “nr” for the examples for constructing
combinations of problem cards general rules
spaces. o {0, 1, #} for bit Analogical reasoning
Because the candidate strings • Idea: if two situations
elimination algorithm perform Explanation based learning are similar in some
breadth-first search, it can be • Idea: can learn better respects, then they will
inefficient. when the background probably be in others
Inductive Bias theory is known • Define the source of an
To further reduce the • Use the domain theory analogy to be a problem
size of the concept space. to explain the instances solution. It is a theory
Such biases constrain the taught that is relatively well
language used to Generalize the explanation to understood.
represent concepts. come up with a “learned rule • The target of an analogy
>Usually the space of learning The EBL algorithm is a theory that is not
algorithms is very large Initialize hypothesis = { } completely understood.
> Consider learning a For each positive training • Analogy constructs a
classification of bit strings example not covered by mapping between
– A classification is hypothesis: corresponding elements
simply a subset of all 1. Explain how training example of the target and the
possible bit strings satisfies source.
– If there are n bits there target concept, in terms of
are 2^n possible bit domain theory
strings 2. Analyze the explanation to
– If a set has m elements, determine the
it has 2^m possible most general conditions under
subsets which this
– Therefore there are explanation (proof) holds
2^(2^n) possible