<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Synchronous RVI (sRVI) on the 3-loop problem
Step1:  For $\alpha=1$, sRVI does not converge on the (periodic) 3-loop problem.
Synchronous DVI (sDVI) on the 3-loop problem
<ASSISTANT_TASK:>
Python Code:
alphas = [1.0, 0.999, 0.99, 0.9, 0.7, 0.5, 0.3, 0.1, 0.01, 0.001]
max_iters = 50000
epsilon = 0.001
init_v = np.zeros(env.num_states())
init_r_bar_scalar = 0
convergence_flags = np.zeros(alphas.__len__())
for i, alpha in enumerate(alphas):
    alg = RVI_Evaluation(env, init_v, alpha, ref_idx=0)
    print(f'RVI Evaluation starting alpha:{alpha}', end=' ')
    convergence = run_alg(alg, 'exec_sync', max_iters, epsilon)
    print(f'Converged? {convergence}')
    convergence_flags[i] = convergence
plt.figure(figsize=(15, 15))
plt.yticks(np.arange(alphas.__len__()), alphas)
plt.ylabel(r'$\alpha$', rotation=0, labelpad=20)
results = np.array([convergence_flags]).reshape(-1, 1)
plt.imshow(results, cmap='viridis', interpolation='nearest')
plt.colorbar()
plt.clim(0, 1)
plt.title('RVI Evaluation')
plt.show()
plt.close()
Explanation: Synchronous RVI (sRVI) on the 3-loop problem
End of explanation
betas = [1.0, 0.999, 0.99, 0.9, 0.7, 0.5, 0.3, 0.1, 0.01, 0.001]
convergence_flags = np.zeros((alphas.__len__(), betas.__len__()))
for i, alpha in enumerate(alphas):
    for j, beta in enumerate(betas):
        alg = DVI_Evaluation(env, init_v, init_r_bar_scalar, alpha, beta)
        print(f'DVI Evaluation starting alpha:{alpha} beta:{beta}', end=' ')
        convergence = run_alg(alg, 'exec_sync', max_iters, epsilon)
        print(f'Converged? {convergence}')
        convergence_flags[i][j] = convergence
plt.figure(figsize=(15, 15))
plt.yticks(np.arange(alphas.__len__()), alphas)
plt.ylabel(r'$\alpha$', rotation=0, labelpad=20)
plt.xlabel(r'$\beta$')
plt.xticks(np.arange(betas.__len__()), betas)
plt.imshow(convergence_flags, cmap='viridis', interpolation='nearest')
plt.colorbar()
plt.clim(0, 1)
plt.title('DVI Evaluation')
plt.show()
plt.close()
Explanation: For $\alpha=1$, sRVI does not converge on the (periodic) 3-loop problem.
Synchronous DVI (sDVI) on the 3-loop problem
End of explanation
<END_TASK>