在學(xué)習(xí)的過程中，抓住基本概念是非常重要的，這樣可以防止自己陷入某些細(xì)節(jié)中無法自拔，可以讓自己站在比較宏觀的層面上看待問題。 更多見：iii.run

機(jī)器學(xué)習(xí)（Machine Learning）

• Coursera上機(jī)器學(xué)習(xí)課程的介紹中，第一句話是：
  　**Machine learning is the science of getting computers to act without being explicitly programmed. **
  　機(jī)器學(xué)習(xí)是讓電腦自己學(xué)會(huì)如何處理問題，而不需要人們刻意的編程。
• Andrew Ng老師的講義中（Coursera）給出的定義：
  Tom Mitchell provides a more modern definition: “A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.”
  Example: playing checkers.
  　· E = the experience of playing many games of checkers
  　· T = the task of playing checkers.
  　· P = the probability that the program will win the next game.
• 周志華的《機(jī)器學(xué)習(xí)》中定義：
  機(jī)器學(xué)習(xí)正是這樣一門學(xué)科，它致力于研究如何通過計(jì)算的手段，利用經(jīng)驗(yàn)來改善系統(tǒng)自身的性能。在計(jì)算機(jī)系統(tǒng)中，“經(jīng)驗(yàn)”通常以“數(shù)據(jù)”的形式存在，因此，機(jī)器學(xué)習(xí)所研究的主要內(nèi)容，是關(guān)于在計(jì)算機(jī)上從數(shù)據(jù)中產(chǎn)生“模型”（model）的算法，即“學(xué)習(xí)算法”（learning algorithm）。有了學(xué)習(xí)算法，我們把經(jīng)驗(yàn)數(shù)據(jù)提供給它，它就能基于這些數(shù)據(jù)產(chǎn)生模型；在面對(duì)新的情況時(shí)，模型會(huì)給我們提供相應(yīng)的判斷。

有監(jiān)督學(xué)習(xí)（Supervised Learning）

• 維基百科
  Supervised learning is the machine learningtask of inferring a function from labeled training data.
  The training data consist of a set of training examples. In supervised learning, each example is a pair consisting of an input object (typically a vector) and a desired output value (also called the supervisory signal). A supervised learning algorithm analyzes the training data and produces an inferred function, which can be used for mapping new examples. An optimal scenario will allow for the algorithm to correctly determine the class labels for unseen instances. This requires the learning algorithm to generalize from the training data to unseen situations in a "reasonable" way.

**Example: **
　Given data about the size of houses on the real estate market, try to predict their price. Price as a function of size is a continuous output, so this is a regression problem.
　We could turn this example into a classification problem by instead making our output about whether the house “sells for more or less than the asking price.” Here we are classifying the houses based on price into two discrete categories

無監(jiān)督學(xué)習(xí)（Unsupervised Learning）

維基百科：
　Unsupervised learning is the machine learning task of inferring a function to describe hidden structure from unlabeled data. Since the examples given to the learner are unlabeled, there is no error or reward signal to evaluate a potential solution. This distinguishes unsupervised learning from supervised learning and reinforcement learning. 　Unsupervised learning is closely related to the problem of density estimation in statistics.[1] However unsupervised learning also encompasses many other techniques that seek to summarize and explain key features of the data. Many methods employed in unsupervised learning are based on data mining methods used to preprocess data.

Example:
　Clustering: Take a collection of 1000 essays written on the US Economy, and find a way to automatically group these essays into a small number that are somehow similar or related by different variables, such as word frequency, sentence length, page count, and so on.
　Associative: Suppose a doctor over years of experience forms associations in his mind between patient characteristics and illnesses that they have. If a new patient shows up then based on this patient’s characteristics such as symptoms, family medical history, physical attributes, mental outlook, etc the doctor associates possible illness or illnesses based on what the doctor has seen before with similar patients. This is not the same as rule based reasoning as in expert systems. In this case we would like to estimate a mapping function from patient characteristics into illnesses.

解釋一下：

首先看什么是學(xué)習(xí)（learning）？一個(gè)成語就可概括：舉一反三。此處以高考為例，高考的題目在上考場(chǎng)前我們未必做過，但在高中三年我們做過很多很多題目，懂解題方法，因此考場(chǎng)上面對(duì)陌生問題也可以算出答案。機(jī)器學(xué)習(xí)的思路也類似：我們能不能利用一些訓(xùn)練數(shù)據(jù)（已經(jīng)做過的題），使機(jī)器能夠利用它們（解題方法）分析未知數(shù)據(jù)（高考的題目）？

最簡單也最普遍的一類機(jī)器學(xué)習(xí)算法就是分類（classification）。對(duì)于分類，輸入的訓(xùn)練數(shù)據(jù)有特征（feature），有標(biāo)簽（label）。所謂的學(xué)習(xí)，其本質(zhì)就是找到特征和標(biāo)簽間的關(guān)系（mapping）。這樣當(dāng)有特征而無標(biāo)簽的未知數(shù)據(jù)輸入時(shí)，我們就可以通過已有的關(guān)系得到未知數(shù)據(jù)標(biāo)簽。

在上述的分類過程中，如果所有訓(xùn)練數(shù)據(jù)都有標(biāo)簽，則為有監(jiān)督學(xué)習(xí)（supervised learning）。如果數(shù)據(jù)沒有標(biāo)簽，顯然就是無監(jiān)督學(xué)習(xí)（unsupervised learning）了，也即聚類（clustering）。

目前分類算法的效果還是不錯(cuò)的，但相對(duì)來講，聚類算法就有些慘不忍睹了。確實(shí)，無監(jiān)督學(xué)習(xí)本身的特點(diǎn)使其難以得到如分類一樣近乎完美的結(jié)果。這也正如我們?cè)诟咧凶鲱}，答案（標(biāo)簽）是非常重要的，假設(shè)兩個(gè)完全相同的人進(jìn)入高中，一個(gè)正常學(xué)習(xí)，另一人做的所有題目都沒有答案，那么想必第一個(gè)人高考會(huì)發(fā)揮更好，第二個(gè)人會(huì)發(fā)瘋。

這時(shí)各位可能要問，既然分類如此之好，聚類如此之不靠譜，那為何我們還可以容忍聚類的存在？因?yàn)樵趯?shí)際應(yīng)用中，標(biāo)簽的獲取常常需要極大的人工工作量，有時(shí)甚至非常困難。例如在自然語言處理（NLP）中，Penn Chinese Treebank在2年里只完成了4000句話的標(biāo)簽……

這時(shí)有人可能會(huì)想，難道有監(jiān)督學(xué)習(xí)和無監(jiān)督學(xué)習(xí)就是非黑即白的關(guān)系嗎？有沒有灰呢？Good idea。灰是存在的。二者的中間帶就是半監(jiān)督學(xué)習(xí)（semi-supervised learning）。對(duì)于半監(jiān)督學(xué)習(xí)，其訓(xùn)練數(shù)據(jù)的一部分是有標(biāo)簽的，另一部分沒有標(biāo)簽，而沒標(biāo)簽數(shù)據(jù)的數(shù)量常常極大于有標(biāo)簽數(shù)據(jù)數(shù)量（這也是符合現(xiàn)實(shí)情況的）。隱藏在半監(jiān)督學(xué)習(xí)下的基本規(guī)律在于：數(shù)據(jù)的分布必然不是完全隨機(jī)的，通過一些有標(biāo)簽數(shù)據(jù)的局部特征，以及更多沒標(biāo)簽數(shù)據(jù)的整體分布，就可以得到可以接受甚至是非常好的分類結(jié)果。（此處大量忽略細(xì)節(jié)）

因此，learning家族的整體構(gòu)造是這樣的：
有監(jiān)督學(xué)習(xí)（分類，回歸）
?
半監(jiān)督學(xué)習(xí)（分類，回歸），transductive learning（分類，回歸）
?
半監(jiān)督聚類（有標(biāo)簽數(shù)據(jù)的標(biāo)簽不是確定的，類似于：肯定不是xxx，很可能是yyy）
?
無監(jiān)督學(xué)習(xí)（聚類）

總結(jié)

在Coursera上Andrew Ng的課程中，主要涉及：

• Supervised Learning，包括Linear Regression、Logistic Regression、Neural Networks、Support Vector Machines；
• Unsupervised Learning，包括Clustering、Dimensionality Reduction、Anomaly Detection、Recommender Systems。

參考資料

Coursera - Machine learning( Andrew Ng) https://www.coursera.org/learn/machine-learning
什么是無監(jiān)督學(xué)習(xí)？ https://www.zhihu.com/question/23194489