Introduction

Suppose you have two kinds of points scattered on a page — say, blue and orange — and you want a single straight line that keeps the colors apart. If the colors don't overlap, there are usually infinitely many lines that do the job. Which one should you trust?

A Support Vector Machine (SVM) gives a crisp answer: pick the line that leaves the widest empty street between the two groups. Don't just barely separate them — separate them with as much breathing room as possible on both sides.

The surprise is how few points actually decide that street. Most of your data could be moved around freely and the boundary wouldn't budge. Only the handful of points sitting right on the curb — the support vectors — hold the whole thing in place. That single idea, maximum margin, turns a vague "draw a good line" into a precise, solvable problem.

Find the Widest Street

Below are two classes of points, blue and orange. The shaded band is the street — the widest gap that separates the two colors — and the dashed lines are its curbs. The points touching a curb are the support vectors, highlighted with a ring.

<p class="hint">Two classes of points: <b class="bl">blue</b> and <b class="og">orange</b>. The shaded band is the widest <b>street</b> separating them; the points on its curbs, ringed in black, are the <b>support vectors</b>. Drag any point and watch it re-shape.</p>
<canvas id="cv" width="460" height="340"></canvas>
<div class="status" id="status"></div>
<div class="btns">
  <button id="reset" type="button">Reset points</button>
  <button id="add" type="button" class="ghost">Add blue / orange</button>
</div>

* { box-sizing: border-box; }
body { font-family: system-ui, sans-serif; color: #222; margin: 0; }
.hint { font-size: .9rem; color: #444; margin: 0 0 .6rem; line-height: 1.45; }
.hint .bl { color: #2a64d6; }
.hint .og { color: #e08a1e; }
#cv { border: 1px solid #cdd9e3; border-radius: 10px; background: #f7fafc;
      touch-action: none; max-width: 100%; cursor: grab; }
#cv.drag { cursor: grabbing; }
.status { font-size: 1rem; font-weight: 600; margin: .6rem 0; min-height: 1.4em; }
.status.ok { color: #0a7d33; }
.status.bad { color: #c92f3c; }
.btns { display: flex; gap: .5rem; flex-wrap: wrap; }
button { font: 600 14px system-ui, sans-serif; padding: .45rem .9rem; border: 1px solid #1d3557;
         background: #1d3557; color: #fff; border-radius: 8px; cursor: pointer; }
button.ghost { background: #fff; color: #1d3557; }

const fmt = (s, v) => s.replace(/%(\w+)%/g, (_, k) => String(v[k] ?? ''));
// Maximum-margin (hard-margin) SVM in 2D, computed geometrically.
// The maximum-margin separator is the perpendicular bisector of the shortest
// segment between the convex hulls of the two classes. We find the closest
// pair of points (one in each hull) by alternating projections: repeatedly
// project one anchor onto the other class's hull until it stops moving.
// The support vectors are the original points whose hull edge the closest points land on. Convex, single global optimum (Cortes & Vapnik, 1995).
const cv = document.getElementById('cv'), ctx = cv.getContext('2d');
const W = cv.width, H = cv.height, R = 9;
let A = [[90,90],[120,200],[80,260],[150,120]];   // blue  (class +1)
let B = [[350,110],[320,240],[380,190],[300,160]]; // orange (class -1)
const A0 = A.map(p=>p.slice()), B0 = B.map(p=>p.slice());

const sub=(p,q)=>[p[0]-q[0],p[1]-q[1]];
const dot=(p,q)=>p[0]*q[0]+p[1]*q[1];
const dist2=(p,q)=>{const d=sub(p,q);return dot(d,d);};

// Closest point to T on segment PQ.
function closestOnSeg(P,Q,T){
  const pq=sub(Q,P); const l2=dot(pq,pq);
  if(l2===0) return P.slice();
  let t=dot(sub(T,P),pq)/l2; t=Math.max(0,Math.min(1,t));
  return [P[0]+t*pq[0], P[1]+t*pq[1]];
}
// Closest point to T on the convex hull (boundary) of set S — approximated by
// the closest point over all hull edges; for the separating-distance problem
// the nearest point of the hull to an external T lies on its boundary.
function hullPoints(S){
  if(S.length<3) return S.map(p=>p.slice());
  const pts=S.map(p=>p.slice()).sort((a,b)=>a[0]-b[0]||a[1]-b[1]);
  const cross=(o,a,b)=>(a[0]-o[0])*(b[1]-o[1])-(a[1]-o[1])*(b[0]-o[0]);
  const lower=[]; for(const p of pts){ while(lower.length>=2 && cross(lower[lower.length-2],lower[lower.length-1],p)<=0) lower.pop(); lower.push(p); }
  const upper=[]; for(let i=pts.length-1;i>=0;i--){ const p=pts[i]; while(upper.length>=2 && cross(upper[upper.length-2],upper[upper.length-1],p)<=0) upper.pop(); upper.push(p); }
  lower.pop(); upper.pop(); return lower.concat(upper);
}
function closestOnHull(S,T){
  const h=hullPoints(S); let best=h[0],bd=Infinity;
  for(let i=0;i<h.length;i++){ const c=closestOnSeg(h[i],h[(i+1)%h.length],T); const d=dist2(c,T); if(d<bd){bd=d;best=c;} }
  return best;
}
// Alternating projection between the two hulls -> closest pair (a* in hull A, b* in hull B).
function closestPair(){
  let a=A[0].slice();
  for(let it=0;it<60;it++){
    const b=closestOnHull(B,a);
    const a2=closestOnHull(A,b);
    if(dist2(a2,a)<1e-7){ a=a2; return [a,b]; }
    a=a2;
  }
  const b=closestOnHull(B,a); return [a,b];
}

// A class is linearly separable iff the hulls don't intersect, i.e. the closest
// pair has positive distance.
function solve(){
  const [a,b]=closestPair();
  const w=sub(b,a);                 // normal, pointing from A toward B
  const wn=Math.hypot(w[0],w[1]);
  const mid=[(a[0]+b[0])/2,(a[1]+b[1])/2];
  const margin=wn;                  // full street width (a* to b*)
  const sep = wn>1e-3;
  return {a,b,w,wn,mid,margin,sep};
}
// signed side of a point relative to the boundary line through mid with normal w
function side(p,sol){ return dot(sub(p,sol.mid),sol.w); }

function draw(){
  ctx.clearRect(0,0,W,H);
  const s=solve();
  if(s.sep){
    const ux=s.w[0]/s.wn, uy=s.w[1]/s.wn;          // unit normal
    const px=-uy, py=ux;                            // boundary direction
    const half=s.wn/2;                              // half-street
    const L=900;
    // street band (two curbs through a and b)
    ctx.fillStyle='rgba(120,160,210,0.16)';
    ctx.beginPath();
    ctx.moveTo(s.a[0]-px*L, s.a[1]-py*L);
    ctx.lineTo(s.a[0]+px*L, s.a[1]+py*L);
    ctx.lineTo(s.b[0]+px*L, s.b[1]+py*L);
    ctx.lineTo(s.b[0]-px*L, s.b[1]-py*L);
    ctx.closePath(); ctx.fill();
    // curbs (dashed)
    ctx.setLineDash([6,5]); ctx.strokeStyle='#7f93ad'; ctx.lineWidth=1.5;
    for(const c of [s.a,s.b]){ ctx.beginPath(); ctx.moveTo(c[0]-px*L,c[1]-py*L); ctx.lineTo(c[0]+px*L,c[1]+py*L); ctx.stroke(); }
    // boundary (solid)
    ctx.setLineDash([]); ctx.strokeStyle='#1d3557'; ctx.lineWidth=2.2;
    ctx.beginPath(); ctx.moveTo(s.mid[0]-px*L,s.mid[1]-py*L); ctx.lineTo(s.mid[0]+px*L,s.mid[1]+py*L); ctx.stroke();
  }
  // identify support vectors: original points within ~R of a curb
  const isSV=(p)=>{ if(!s.sep) return false; const sa=Math.abs(side(p,s)-side(s.a,s)); const sb=Math.abs(side(p,s)-side(s.b,s)); return Math.min(sa,sb) < s.wn*R*0.9; };
  const drawPt=(p,fill,sv)=>{
    ctx.beginPath(); ctx.arc(p[0],p[1],R,0,7); ctx.fillStyle=fill; ctx.fill();
    if(sv){ ctx.lineWidth=3; ctx.strokeStyle='#111'; ctx.stroke(); }
    else { ctx.lineWidth=1.5; ctx.strokeStyle='rgba(0,0,0,.35)'; ctx.stroke(); }
  };
  A.forEach(p=>drawPt(p,'#2a64d6', isSV(p)));
  B.forEach(p=>drawPt(p,'#e08a1e', isSV(p)));

  const st=document.getElementById('status');
  if(s.sep){ st.textContent=fmt('Separable — street width ≈ %margin% px. Drag a support vector (ringed) to change it.', { margin: s.margin.toFixed(0) }); st.className='status ok'; }
  else { st.textContent='Not separable — the classes overlap, so no clean street exists.'; st.className='status bad'; }
}

// dragging
let drag=null;
function pos(e){ const r=cv.getBoundingClientRect(); const t=e.touches?e.touches[0]:e; return [(t.clientX-r.left)*(W/r.width),(t.clientY-r.top)*(H/r.height)]; }
function pick(m){ for(const arr of [A,B]) for(const p of arr) if(dist2(p,m)<=(R+4)*(R+4)) return p; return null; }
function down(e){ const m=pos(e); drag=pick(m); if(drag){ cv.classList.add('drag'); e.preventDefault(); } }
function move(e){ if(!drag) return; const m=pos(e); drag[0]=Math.max(R,Math.min(W-R,m[0])); drag[1]=Math.max(R,Math.min(H-R,m[1])); draw(); e.preventDefault(); }
function up(){ drag=null; cv.classList.remove('drag'); }
cv.addEventListener('mousedown',down); window.addEventListener('mousemove',move); window.addEventListener('mouseup',up);
cv.addEventListener('touchstart',down,{passive:false}); cv.addEventListener('touchmove',move,{passive:false}); cv.addEventListener('touchend',up);

document.getElementById('reset').addEventListener('click',()=>{ A=A0.map(p=>p.slice()); B=B0.map(p=>p.slice()); draw(); });
document.getElementById('add').addEventListener('click',()=>{ A.push([60+Math.random()*120,60+Math.random()*220]); B.push([300+Math.random()*120,60+Math.random()*220]); draw(); });
draw();
Result
Console

Drag any point and watch the street re-shape itself. Notice two things. First, dragging an interior point usually changes nothing — only the support vectors matter. Second, push a point across the gap and the colors stop being separable: the SVM tells you there is no clean street at all. Maximizing the margin is the whole game.

The Real Complexity

Here is the good news that made SVMs famous. Finding the maximum-margin line is not one of the intractable problems.

  • It is a convex problem. Maximizing the margin can be written as a quadratic program: minimize a smooth bowl-shaped function subject to linear constraints. Convex problems have no bad local minima — every valley is the valley.
  • A global optimum, in polynomial time. Standard solvers reach the single best boundary efficiently; specialized methods like SMO scale to large datasets. There is no exponential search and no guessing.
  • Few points matter. At the optimum, only the support vectors have nonzero weight. The boundary is literally a weighted combination of those border points.
  • The kernel trick. Replacing dot products with a kernel lets the same convex machinery draw curved boundaries — separating data that no straight line could — without ever leaving the tractable world.

The method was introduced in its modern soft-margin form by Corinna Cortes and Vladimir Vapnik in 1995. Unlike training a deep neural network, whose loss surface is riddled with local minima, an SVM's objective is convex — so "train it well" and "solve it optimally" are the same thing.

Where It Matters

For two decades the SVM was the default classifier whenever data was scarce but features were many — and it is still a strong baseline today:

  • Text and spam: sorting emails, news and reviews into categories, where each document is a point in a high-dimensional word space.
  • Bioinformatics: classifying genes, proteins and tumor samples, where there are far more measurements than examples.
  • Image and handwriting recognition: a classic early win, recognizing digits and faces before deep nets took over.
  • Anywhere with few labels: because the boundary leans on a few support vectors, SVMs generalize gracefully from small datasets.

The deeper lesson is the maximum-margin principle itself: prefer the decision that stays as far as possible from every example you've seen. That same instinct shows up in PAC learning, where a wide margin is a promise about how well a model will do on data it has never met.

Conclusion

A Support Vector Machine takes a fuzzy instruction — "separate these classes well" — and makes it exact: of all the lines that work, choose the one with the widest margin. That choice is governed by only the support vectors, and finding it is a convex problem we can solve to the global optimum.

That combination is rare and precious. Many learning problems hide nasty, non-convex landscapes; the SVM carved out a corner where the best answer is also the reachable one. Whenever you can frame a decision as "stay as far as possible from the edge," you are thinking like a support vector machine — and you can borrow the same idea behind PAC learning to trust the result.

Share this article

Pick a channel — or use your device's native share sheet.

Comments

Loading comments...

https://www.kipuhub.com/en/article/support-vector-machines/Content licensed under CC BY-NC 4.0.