<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Optimize Trained GPU Model
Types of Optimizations Applied for Inference
Remove training-only operations (checkpoint saving, drop out)
Strip out unused nodes
Remove debug operations
Fold batch normalization ops into weights (super cool)
Round weights
Quantize weights
Graph Transform Tool
https
Step1:  Strip Unused Nodes
Step2:  Fold Constants
Step3:  Fold Batch Normalizations
Must run Fold Constants first!
Step4:  Quantize Weights
Should run Fold Batch Norms first!
Step5:  Perform All Common Optimizations
Step6:  Sort by Execution Order (DAG Topological Order)
Minimizes inference overhead 
Inputs for a node guaranteed to be available
<ASSISTANT_TASK:>
Python Code:
%%bash 
which summarize_graph
%%bash
ls -l /root/models/optimize_me/
%%bash
summarize_graph --in_graph=/root/models/optimize_me/unoptimized_gpu.pb
Explanation: Optimize Trained GPU Model
Types of Optimizations Applied for Inference
Remove training-only operations (checkpoint saving, drop out)
Strip out unused nodes
Remove debug operations
Fold batch normalization ops into weights (super cool)
Round weights
Quantize weights
Graph Transform Tool
https://petewarden.com/2016/12/30/rewriting-tensorflow-graphs-with-the-gtt/
https://github.com/tensorflow/tensorflow/tree/master/tensorflow/tools/graph_transforms
Optimize Models
Summarize Graph Utility
End of explanation
%%bash
transform_graph \
--in_graph=/root/models/optimize_me/unoptimized_gpu.pb \
--out_graph=/root/models/optimize_me/strip_unused_optimized_gpu.pb \
--inputs='x_observed,weights,bias' \
--outputs='add' \
--transforms='
strip_unused_nodes'
%%bash
ls -l /root/models/optimize_me/
%%bash
summarize_graph --in_graph=/root/models/optimize_me/strip_unused_optimized_gpu.pb
%%bash
benchmark_model --graph=/root/models/optimize_me/strip_unused_optimized_gpu.pb --input_layer=weights,bias,x_observed --input_layer_type=float,float,float --input_layer_shape=:: --output_layer=add
Explanation: Strip Unused Nodes
End of explanation
%%bash
transform_graph \
--in_graph=/root/models/optimize_me/unoptimized_gpu.pb \
--out_graph=/root/models/optimize_me/fold_constants_optimized_gpu.pb \
--inputs='x_observed,weights,bias' \
--outputs='add' \
--transforms='
fold_constants(ignore_errors=true)'
%%bash
ls -l /root/models/optimize_me/
%%bash
summarize_graph --in_graph=/root/models/optimize_me/fold_constants_optimized_gpu.pb
%%bash
benchmark_model --graph=/root/models/optimize_me/fold_constants_optimized_gpu.pb --input_layer=x_observed,bias,weights --input_layer_type=float,float,float --input_layer_shape=:: --output_layer=add
Explanation: Fold Constants
End of explanation
%%bash
transform_graph \
--in_graph=/root/models/optimize_me/fold_constants_optimized_gpu.pb \
--out_graph=/root/models/optimize_me/fold_batch_norms_optimized_gpu.pb \
--inputs='x_observed,weights,bias' \
--outputs='add' \
--transforms='
fold_batch_norms
fold_old_batch_norms'
%%bash
ls -l /root/models/optimize_me/
%%bash
summarize_graph --in_graph=/root/models/optimize_me/fold_batch_norms_optimized_gpu.pb
%%bash
benchmark_model --graph=/root/models/optimize_me/fold_batch_norms_optimized_gpu.pb --input_layer=x_observed,bias,weights --input_layer_type=float,float,float --input_layer_shape=:: --output_layer=add
Explanation: Fold Batch Normalizations
Must run Fold Constants first!
End of explanation
%%bash
transform_graph \
--in_graph=/root/models/optimize_me/fold_batch_norms_optimized_gpu.pb \
--out_graph=/root/models/optimize_me/quantized_optimized_gpu.pb \
--inputs='x_observed,weights,bias' \
--outputs='add' \
--transforms='quantize_weights'
%%bash
ls -l /root/models/optimize_me/
%%bash
summarize_graph --in_graph=/root/models/optimize_me/quantized_optimized_gpu.pb
%%bash
benchmark_model --graph=/root/models/optimize_me/quantized_optimized_gpu.pb --input_layer=x_observed,bias,weights --input_layer_type=float,float,float --input_layer_shape=:: --output_layer=add
Explanation: Quantize Weights
Should run Fold Batch Norms first!
End of explanation
%%bash
transform_graph \
--in_graph=/root/models/optimize_me/unoptimized_gpu.pb \
--out_graph=/root/models/optimize_me/fully_optimized_gpu.pb \
--inputs='x_observed,weights,bias' \
--outputs='add' \
--transforms='
add_default_attributes
remove_nodes(op=Identity, op=CheckNumerics)
fold_constants(ignore_errors=true)
fold_batch_norms
fold_old_batch_norms
quantize_weights
quantize_nodes
strip_unused_nodes
obfuscate_names'
%%bash
ls -l /root/models/optimize_me/
%%bash
summarize_graph --in_graph=/root/models/optimize_me/fully_optimized_gpu.pb
%%bash
benchmark_model --graph=/root/models/optimize_me/fully_optimized_gpu.pb --input_layer=weights,x_observed,bias --input_layer_type=float,float,float --input_layer_shape=:: --output_layer=add
Explanation: Perform All Common Optimizations
End of explanation
%%bash
transform_graph \
--in_graph=/root/models/optimize_me/fully_optimized_gpu.pb \
--out_graph=/root/models/optimize_me/sort_by_execution_order_optimized_gpu.pb \
--inputs='x_observed,weights,bias' \
--outputs='add' \
--transforms='
sort_by_execution_order'
%%bash
ls -l /root/models/optimize_me/
%%bash
summarize_graph --in_graph=/root/models/optimize_me/sort_by_execution_order_optimized_gpu.pb
%%bash
benchmark_model --graph=/root/models/optimize_me/sort_by_execution_order_optimized_gpu.pb --input_layer=weights,x_observed,bias --input_layer_type=float,float,float --input_layer_shape=:: --output_layer=add
Explanation: Sort by Execution Order (DAG Topological Order)
Minimizes inference overhead 
Inputs for a node guaranteed to be available
End of explanation
<END_TASK>