Reinforcement Learning Exercise 5.5

Exercise 5.5 Consider an MDP with a single nonterminal state and a single action that transitions back to the nonterminal state with probability $p$ and transitions to the terminal state with probability $1-p$. Let the reward be $+1$ on all transitions, and let $\gamma =1$. Suppose you observe one episode that lasts 10 steps, with a return of 10. What are the first-visit and every-visit estimators of the value of the nonterminal state?

For the first-visit estimator, only the first visit of a state is considered. So:
$$\begin{array}{rl}V(S_{nonterminal})& =G(S_0)\\ & =1\cdot p+0\cdot (1-p)\\ & =p\end{array}$$
For the every-visit estimator, every state is considered:
$$\begin{array}{rl}V(S_{noterminal})& =G(S_0)+G(S_1)+\cdots +G_{(}{S}_{10})\\ & =[1\cdot p+0\cdot (1-p)]+\gamma [1\cdot p^2+0\cdot (1-p)]+\cdots +{\gamma }^9[1\cdot p^{10}+0\cdot (1-p)]\\ & =p+p^2+\cdots +p^{10}\\ & =\frac{p}{1-p}(p^{10}-1)\end{array}$$