Weak Learning DNF under uniform d by cuiliqing


									Weak Learning DNF under uniform
• A parity function weakly approximates f
  – Find this function

• KM algorithm
  – Form a tree, pruning a node if there are no
    large coefficients starting with that substring
         Strong learning: Boosting
• Recall Freund’s algo
   – Construct weak hypotheses h1,h2,h3…
   – At step i:
     Distribution Di: more weight to x on which hi,…,hi-1 were wrong
     Form hypo hi on Di
   – Combine hypotheses using some rule

• Does parity approximate f on Di?
   – Yes.. Answer on next slide
• Needs a distribution-independent weak learner
   – We don’t have one for DNF
         Strong learning DNF
• Let f be a DNF having s-terms
• Lemma: f has a fourier coefficient of value
  at least 1/(2s+1)
           Can we tweak KM?
• Cool fact: KM works for real-valued
  functions as well

• Idea: Construct function g such that

• Depends on L(g) in running time
      Converting (f,D) to (g,U)
• Notice that            on dist D is same as
                 on uniform

• We can learn           on U!!

• Need MQ oracle for g => MQ oracle for D
• L(g) should not be too large
           “D close to uniform”
An appropriate Boosting algorithm
• Final hypothesis – majority of his
• Di = D(x)i(x)  has to be normalized
    i(x)  prob that hypotheses are almost equally
    divided over x
• Stop if Dis become too small

• Notice: easily computable
• Close to uniform – within a small factor

To top