2018-04-24 开胃学习数学系列 - Entropy

Problem 1

Use the dual objective discussed in class to solve the following maximum entropy optimization problems.
Find the the maximum entropy (ME) distribution without any prior distributions (equivalent to uniform priors) with the following constraints:
Hint:

• this problem illustrate that the importance of the prior distributions.
• You can use the me package as part of the class library to solve this problem. You will get extra credit if you can implement your own version of the ME optimizations. If you choose to implement your own, beware that the exp() function may overflow, so you need to apply a cap before calling exp().
• for #1. you should try a discretization of a wide range first, for example (-6, 6) with small sampling steps
• the Q-Q plot is an effective way to show whether two distributions are similar. Google search Q-Q plot if you don't know what it is.

1. with constraints of [x]=0 and [x^2]=1; show that the resulting distribution is indeed normal

使用课堂讨论的dual objective 来解决以下最大熵 优化问题。

找到没有任何先验分布（相当于统一先验，equivalent to uniform priors）的最大熵（ME）分布，并具有以下约束条件：

问题2

本练习说明误差界限改善了ME方法的 稳健性

很多时候，我们在实践中遇到了conflicting constraints 的相互矛盾的约束条件：

1. try to solve the ME optimization with standard normal as the prior distribution, with the following conflicting skewness constraints of E[x3]=−2 and E[x3]=2.
   尝试使用标准正态分布作为先验分布来解决ME优化，其中conflicting skewness constraints : E [x^3] = - 2和E [x^3] = 2 冲突skewness约束

Show that the numerical optimizer will fail to find a solution if we don't allow error bounds (ie, like the bid/ask discussed in class);
如果我们不允许误差界限 error bounds ，最优化无法找到答案。

check your implementation so that it fails gracefully
(i.e., it should not stop from running the rest of your ipython notebook cells when "Run All").

show that you can find a solution by introducing error bounds, and shift the distribution to skew left or right by adjusting the error bounds related to these donstraints.

检查你的实现，使它失败（也就是说，它不应该在“全部运行”时停止运行其他ipython笔记本单元）。

表明您可以通过引入误差界限来找到解决方案，并通过调整与这些限制相关的误差界限来将分布转换为向左或向右倾斜。

c