深度知识追踪（Deep Knowledge Tracing）论文学习（简要归纳）

深度知识追踪（Deep Knowledge Tracing）

---

Chris Piech∗ , Jonathan Spencer∗ , Jonathan Huang∗‡, Surya Ganguli∗ , Mehran Sahami∗ , Leonidas Guibas∗ , Jascha Sohl-Dickstein∗†   ∗Stanford University,  †Khan Academy,  ‡Google发布正在：NIPS'15 （人工智能A会）Neural Information Processing Systems

Contributions

A novel application of recurrent neural networks (RNN) to tracing student knowledge. (Model Introduction) 

Demonstration that our model does not need expert annotations. (Previous Work)  

A 25% gain in AUC over the best previous result. (Experimental results)  

Power a number of other applications. (Other Applications)

Knowledge Tracing  

Def:  Knowledge tracing is the task of modelling student knowledge over time so that we can accurately predict how students will perform on future interactions.

知识追踪是基于学生行为序列进行建模，预测学生对知识的掌握程度。知识追踪是构建自适应教育系统的核心和关键。在自适应的教育系统中，无论是做精准推送，学生学习的路径规划或知识图谱的构建，第一步都是能够精准预测学生对知识的掌握程度。
Usually by observing the correctness of doing exercises.

 知识追踪问题可以描述为： 给定一学生的观测序列  x 0 ,……, x t 预测下次表现 x t+1 ，通常 x t ={ q t , a t } xt={qt,at} ，其中 q t qt 代表回答的问题成分（如对应的知识点）， a t at 代表对应的回答是否正确，通常 a t ={0,1} at={0,1} 。上图描述了一个学生在八年级数学中的知识追踪结果可视化展示。

Motivation

• Develop computer-assisted education by building models of large scale student trace data on MOOCs.
• Resources can be suggested to students based on their individual needs.
• Content which is predicted to be too easy or too hard can be skipped or delayed.
• Formal testing is no longer necessary if a student’s ability undergoes continuous assessment
• The knowledge tracing problem is inherently difficult. Most previous work in education relies on first order Markov models with restricted functional forms.

模型 Neural Networks(RNN)

Long Short Term Memory (LSTM)

 

Previous Work 

Bayesian Knowledge Tracing (BKT)

Standard Bayesian Knowledge Tracing (BKT)  (1995)

​​​​

Extensions: Contextualization of guessing and slipping estimates Estimating prior knowledge for individual Learners Estimating problem difficulty

 Drawbacks: The binary representation of student understanding may be unrealistic.
The meaning of the hidden variables and their mappings onto exercises can be ambiguous, rarely meeting the model’s expectation of a single concept per exercise.
The binary response data used to model transitions imposes a limit on the kinds of exercises that can be modeled.
 

 Learning Factors Analysis (LFA)

SPARse Factor Analysis (SPARFA) (JMLR 2014)SPARse Factor Analysis (SPARFA) (JMLR 2014)

The probabilities that the learners answer the questions correctly: +  

Three observations: Typical educational domains of interest involve only a small number of key concept.
Each question involves only a small subset of the abstract concepts.
The entries of should be non-negative.
Drawback: They are both more restricted in functional form and more expensive (due to inference of latent variables) than the method we present here.
 

 Performance Factors Analysis (PFA)  

SPARse Factor Analysis (SPARFA)

RNN Model

In contrast to hidden Markov models as they appear in education, which are also dynamic, RNNs have a high dimensional, continuous, representation of latent state.
A notable advantage of the richer representation of RNNs is their ability to use information from an input in a prediction at a much later point in time.
 

 

Experiment results

Expectimax  V.S. mixing (exercises from different topics are intermixed) 、blocking (students answer series of exercises of the same Type).
Tested different curricula for selecting exercises on a subset of five concepts over the span of 30 exercises from the ASSISTment dataset.

 

Future Work

Incorporate other features as inputs (such as time taken) Explore other educational impacts (such as hint generation, dropout prediction) Validate hypotheses posed in education literature (such as spaced repetition, modeling how students forget) To track knowledge over more complex learning activities 

模型优缺点：

优点： 能够反应长时间的知识关系，基于RNN的特性能够根据学生近期学习表现进行预测（近因效应），也能根据实际学生学习路径进行建模； 能够对复杂的知识点之间的联系进行建模，如构建知识图谱； 能够处理多知识成分的问题。
缺点： 模型无法重构输入，即输入某一知识成分答题错误，模型对该知识成分的预测反而是正确。
在时间序列上，学生对知识点的掌握程度不具有连续一致性，波动情况较大