<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 MODELED.netconf
github.com/modeled/modeled.netconf
Highly Pythonized NETCONF and YANG
Automagically create YANG
  modules and data containers from
  MODELED Python classes
  >>>
Simply turn Python methods into
  NETCONF/YANG RPC methods
  using decorators
  >>>
Directly start NETCONF servers from modeled YANG modules
  >>>
TODO
Step7:  To install in development mode
Step8:  To check if the Turing Machine works, it needs an actual program.
I took it from the pyang tutorial again.
It's a very simple program for adding to numbers in unary notation,
separated by a 0.
It can easily be defined YAML.
If you haven't installed pyyaml yet
Step9:  Instantiate the Turing Machine with the loaded program
Step10:  And set the initial state for computing unary 1 + 2
Step11:  The tape string gets automatically converted to a list,
because TuringMachine.tape is defined as a list member
Step12:  Ready for turning on the Turing Machine
Step13:  Final state is reached. Result is unary 3. Seems to work!
YANGifying the Turing Machine
Creating a YANG module from the modeled TuringMachine class
is now quite simple. Just import the modeled YANG module adapter class
Step14:  And plug it to the TuringMachine.
This will create a new class which will be derived
from both the YANG module adapter and the TuringMachine class
Step15:  It also has a class attribute referencing the original modeled class
Step16:  BTW
Step17:  But let's take look at the really useful features now.
The adapted class dynamically provides .to_...() methods
for every pyang output format plugin
which you could pass to the pyang command's -f flag.
Calling such a method will programmatically
create a pyang.statement.Statement tree
(which pyang does internally on loading an input file)
according to the typed members of the adapted modeled class.
Every .to_...() method takes optional
revision date and XML prefix and namespace arguments.
If no revision is given,
the current date will be used.
The adapted class will be mapped to a YANG module
and its main data container definition.
Module and container name will be generated from the name
of the adapted modeled class
by decapitalizing and joining its name parts with hyphens.
YANG leaf names will be generated from modeled member names
by replacing underscores with hyphens.
list and dict members will be mapped to YANG list definitions.
If members have other modeled classes as types,
sub-containers will be defined.
Type mapping is very simple in this early project stage.
Only int and str are supported
and no YANG typedefs are used.
All containers and their contents are defined configurable
(with write permissions).
That will change soon...
The result is a complete module definition text in the given format,
like default YANG
Step18:  Or XMLified YIN
Step19:  Since the modeled YANG module
is derived from the adapted TuringMachine class,
it can still be instantiated and used in the same way
Step20:  Adding RPC methods
The above modeled YANG module is not very useful
without some RPC methods for controlling the Turing Machine via NETCONF.
MODELED.netconf offers a simple @rpc decorator
for defining them
Step23:  The following RPC definitions are again designed
according to the pyang tutorial.
Since those RPC methods are NETCONF/YANG specific,
they are defined after the modeled YANG adaption.
The simplest way is to derive a new class for that purpose
Step24:  Now the .to_yang() conversion also includes the rpc definitions,
with descriptions taken from the Python methods' __doc__ strings,
and rpc and input leaf names automatically
created from the Python method and argument names
by replacing underscores with hyphens again
Step25:  Now is a good time to verify if that's really correct YANG.
Just write it to a file
Step26:  And feed it to the pyang command.
Since the pyang turorial also produces
a tree format output from its YANG Turing Machine,
I also do it here for comparison
(!... runs external programs in IPython)
Step27:  No errors. Great!
From modeled YANG modules to a NETCONF service
Finally! Time to run a Turing Machine NETCONF server...
First create an instance of the final Turing Machine class
with RPC method definitions
Step28:  Currently only serving NETCONF over
SSH is supported.
An SSH service needs a network port and user authentication credentials
Step29:  And it needs an SSH key.
If you don't have any key lying around,
the UNIX tool ssh-keygen from
OpenSSH
(or Windows tools like
PuTTY)
can generate one for you.
Just name the file key
Step30:  And that's it! The created server is an instance of Python
netconf project's
NetconfSSHServer class.
The server's internals run in a separate thread,
so it doesn't block the Python script.
We can just continue with creating a NETCONF client
which talks to the server.
Let's directly use NetconfSSHSession
from the netconf project for now.
The Pythonic client features of MODELED.netconf are not implemented yet,
but they will also be based on netconf.
Step31:  Now the Turing Machine can be remotely initialized
with a NETCONF RPC call.
Let's compute unary 2 + 3 this time.
Normally this would also need the Turing Machine's XML namespace,
but namspace handling is not properly supported yet
by MODELED.netconf
Step32:  The tape will be set accordingly
Step33:  Now run the Turing Machine via RPC
<ASSISTANT_TASK:>
Python Code:
import modeled.netconf
modeled.netconf.__requires__
Explanation: MODELED.netconf
github.com/modeled/modeled.netconf
Highly Pythonized NETCONF and YANG
Automagically create YANG
  modules and data containers from
  MODELED Python classes
  >>>
Simply turn Python methods into
  NETCONF/YANG RPC methods
  using decorators
  >>>
Directly start NETCONF servers from modeled YANG modules
  >>>
TODO:
Proper RPC namespace handling
Handle default NETCONF RPC methods like &lt;get&gt; or &lt;get-config&gt;
Create NETCONF servers with multiple modeled YANG modules
Automagically create Pythonic NETCONF clients from YANG definitions
WARNING: This project is in early alpha state
and therefore not production ready.
ABOUT this README:
It contains setup instructions
  and a tutorial.
It was automatically created from
  IPython notebook README.ipynb.
  You can view the notebook
  online.
The internal links don't work on Bitbucket.
Setup
Just use pip
to install the latest release from PyPI:
pip install modeled.netconf
It automatically installs all runtime requirements:
End of explanation
import modeled
from modeled import member
class Input(modeled.object):
    The input part of a Turing Machine program rule.
    
    state = member[int]()
    symbol = member[str]()
class Output(modeled.object):
    The output part of a Turing Machine program rule.
    
    state = member[int]()
    symbol = member[str]()
    head_move = member[str]['L', 'R']()
class Rule(modeled.object):
    A Turing Machine program rule.
    
    input = member[Input]()
    output = member[Output]()
    def __init__(self, input, output):
        Expects both `input` and `output` as mappings.
        
        self.input = Input(
            # modeled.object.__init__ supports **kwargs
            # for initializing modeled.member values
            **dict(input))
        self.output = Output(**dict(output))
class TuringMachine(modeled.object):
    state = member[int]()
    head_position = member[int]()
    # the list of symbols on the input/output tape
    tape = member.list[str](indexname='cell', itemname='symbol')
    # the machine program as named rules
    program = member.dict[str, Rule](keyname='name')
    def __init__(self, program):
        Create a Turing Machine with the given `program`.
        
        program = dict(program)
        for name, (input, output) in program.items():
            self.program[name] = Rule(input, output)
    def run(self):
        Start the Turing Machine.
        
        - Runs until no matching input part for current state and tape symbol
          can be found in the program rules.
        
        self.log = " %s  %d\n" % (''.join(self.tape), self.state)
        while True:
            pos = self.head_position
            if 0 <= pos < len(self.tape):
                symbol = self.tape[pos]
            else:
                symbol = None
            for name, rule in self.program.items():
                if (self.state, symbol) == (rule.input.state, rule.input.symbol):
                    self.log += "%s^%s --> %s\n" % (
                        ' ' * (pos + 1),
                        ' ' * (len(self.tape) - pos),
                        name)
                    if rule.output.state is not None:
                        self.state = rule.output.state
                    if rule.output.symbol is not None:
                        self.tape[pos] = rule.output.symbol
                    self.head_position += {'L': -1, 'R': 1}[rule.output.head_move]
                    self.log += " %s  %d\n" % (''.join(self.tape), self.state)
                    break
            else:
                break
Explanation: To install in development mode:
pip install -e /path/to/repository/
From a MODELED class to a YANG module
MODELED.netconf is based on my
MODELED framework,
which provides tools for defining Python classes
with typed members and methods,
quite similar to Django database models,
but with a more general approach.
Those modeled classes can then automagically be mapped
to data serialization formats, databases,
GUI frameworks, web frameworks, or whatever,
using the integrated modeled.Adapter system.
The MODELED framework is still in a late alpha stage,
needs some internal refactoring, and lacks documentation,
but I am actively working on this.
The basic principles should nevertheless become visible during the following example.
A MODELED Turing Machine
Since MODELED.netconf uses pyang
for auto-generating YANG definitions from modeled classes,
I decided to resemble the Turing Machine example from the
pyang tutorial...
a bit more simplified and with some little structural and naming changes...
however... below is a modeled Turing Machine implementation:
End of explanation
%%file turing-machine-program.yaml
left summand:
  - {state:    0, symbol:    1}
  - {state: null, symbol: null, head_move: R}
separator:
  - {state:    0, symbol:    0}
  - {state:    1, symbol:    1, head_move: R}
right summand:
  - {state:    1, symbol:    1}
  - {state: null, symbol: null, head_move: R}
right end:
  - {state:    1, symbol: null}
  - {state:    2, symbol: null, head_move: L}
write separator:
  - {state:    2, symbol:    1}
  - {state:    3, symbol:    0, head_move: L}
go home:
  - {state:    3, symbol:    1}
  - {state: null, symbol: null, head_move: L}
final step:
  - {state:    3, symbol: null}
  - {state:    4, symbol: null, head_move: R}
import yaml
with open('turing-machine-program.yaml') as f:
    TM_PROGRAM = yaml.load(f)
Explanation: To check if the Turing Machine works, it needs an actual program.
I took it from the pyang tutorial again.
It's a very simple program for adding to numbers in unary notation,
separated by a 0.
It can easily be defined YAML.
If you haven't installed pyyaml yet:
pip install pyyaml
(%%... are IPython magic functions):
End of explanation
tm = TuringMachine(TM_PROGRAM)
Explanation: Instantiate the Turing Machine with the loaded program:
End of explanation
tm.state = 0
tm.head_position = 0
tm.tape = '1011'
Explanation: And set the initial state for computing unary 1 + 2:
End of explanation
tm.tape
Explanation: The tape string gets automatically converted to a list,
because TuringMachine.tape is defined as a list member:
End of explanation
tm.run()
print(tm.log)
Explanation: Ready for turning on the Turing Machine:
End of explanation
from modeled.netconf import YANG
Explanation: Final state is reached. Result is unary 3. Seems to work!
YANGifying the Turing Machine
Creating a YANG module from the modeled TuringMachine class
is now quite simple. Just import the modeled YANG module adapter class:
End of explanation
YANG[TuringMachine].mro()
Explanation: And plug it to the TuringMachine.
This will create a new class which will be derived
from both the YANG module adapter and the TuringMachine class:
End of explanation
YANG[TuringMachine].mclass
Explanation: It also has a class attribute referencing the original modeled class:
End of explanation
YANG[TuringMachine] is YANG[TuringMachine]
Explanation: BTW: the class adaption will be cached,
so every YANG[TuringMachine] operation
will return the same class object:
End of explanation
print(YANG[TuringMachine].to_yang(
    prefix='tm', namespace='http://modeled.netconf/turing-machine'))
Explanation: But let's take look at the really useful features now.
The adapted class dynamically provides .to_...() methods
for every pyang output format plugin
which you could pass to the pyang command's -f flag.
Calling such a method will programmatically
create a pyang.statement.Statement tree
(which pyang does internally on loading an input file)
according to the typed members of the adapted modeled class.
Every .to_...() method takes optional
revision date and XML prefix and namespace arguments.
If no revision is given,
the current date will be used.
The adapted class will be mapped to a YANG module
and its main data container definition.
Module and container name will be generated from the name
of the adapted modeled class
by decapitalizing and joining its name parts with hyphens.
YANG leaf names will be generated from modeled member names
by replacing underscores with hyphens.
list and dict members will be mapped to YANG list definitions.
If members have other modeled classes as types,
sub-containers will be defined.
Type mapping is very simple in this early project stage.
Only int and str are supported
and no YANG typedefs are used.
All containers and their contents are defined configurable
(with write permissions).
That will change soon...
The result is a complete module definition text in the given format,
like default YANG:
End of explanation
print(YANG[TuringMachine].to_yin(
    prefix='tm', namespace='http://modeled.netconf/turing-machine'))
Explanation: Or XMLified YIN:
End of explanation
tm = YANG[TuringMachine](TM_PROGRAM)
tm.state = 0
tm.head_position = 0
tm.tape = '1011'
tm.run()
tm.state, tm.tape
Explanation: Since the modeled YANG module
is derived from the adapted TuringMachine class,
it can still be instantiated and used in the same way:
End of explanation
from modeled.netconf import rpc
Explanation: Adding RPC methods
The above modeled YANG module is not very useful
without some RPC methods for controlling the Turing Machine via NETCONF.
MODELED.netconf offers a simple @rpc decorator
for defining them:
End of explanation
class TM(YANG[TuringMachine]):
    @rpc(argtypes={'tape_content': str})
    # in Python 3 you can also use function annotations
    # and write (self, tape_content: str) below
    # instead of argtypes= above
    def initialize(self, tape_content):
        Initialize the Turing Machine.
        
        self.state = 0
        self.head_position = 0
        self.tape = tape_content
    @rpc(argtypes={})
    def run(self):
        Start the Turing Machine operation.
        
        TuringMachine.run(self)
Explanation: The following RPC definitions are again designed
according to the pyang tutorial.
Since those RPC methods are NETCONF/YANG specific,
they are defined after the modeled YANG adaption.
The simplest way is to derive a new class for that purpose:
End of explanation
TM_YANG = TM.to_yang(
    prefix='tm', namespace='http://modeled.netconf/turing-machine')
print(TM_YANG)
Explanation: Now the .to_yang() conversion also includes the rpc definitions,
with descriptions taken from the Python methods' __doc__ strings,
and rpc and input leaf names automatically
created from the Python method and argument names
by replacing underscores with hyphens again:
End of explanation
with open('turing-machine.yang', 'w') as f:
    f.write(TM_YANG)
Explanation: Now is a good time to verify if that's really correct YANG.
Just write it to a file:
End of explanation
!pyang -f tree turing-machine.yang
Explanation: And feed it to the pyang command.
Since the pyang turorial also produces
a tree format output from its YANG Turing Machine,
I also do it here for comparison
(!... runs external programs in IPython):
End of explanation
tm = TM(TM_PROGRAM)
Explanation: No errors. Great!
From modeled YANG modules to a NETCONF service
Finally! Time to run a Turing Machine NETCONF server...
First create an instance of the final Turing Machine class
with RPC method definitions:
End of explanation
PORT = 12345
USERNAME = 'user'
PASSWORD = 'password'
Explanation: Currently only serving NETCONF over
SSH is supported.
An SSH service needs a network port and user authentication credentials:
End of explanation
server = tm.serve_netconf_ssh(
    port=PORT, host_key='key', username=USERNAME, password=PASSWORD)
Explanation: And it needs an SSH key.
If you don't have any key lying around,
the UNIX tool ssh-keygen from
OpenSSH
(or Windows tools like
PuTTY)
can generate one for you.
Just name the file key:
ssh-keygen -f key
End of explanation
from netconf.client import NetconfSSHSession
client = NetconfSSHSession(
    'localhost', port=PORT, username=USERNAME, password=PASSWORD)
Explanation: And that's it! The created server is an instance of Python
netconf project's
NetconfSSHServer class.
The server's internals run in a separate thread,
so it doesn't block the Python script.
We can just continue with creating a NETCONF client
which talks to the server.
Let's directly use NetconfSSHSession
from the netconf project for now.
The Pythonic client features of MODELED.netconf are not implemented yet,
but they will also be based on netconf.
End of explanation
reply = client.send_rpc(
    '<initialize><tape-content>110111</tape-content></initialize>')
Explanation: Now the Turing Machine can be remotely initialized
with a NETCONF RPC call.
Let's compute unary 2 + 3 this time.
Normally this would also need the Turing Machine's XML namespace,
but namspace handling is not properly supported yet
by MODELED.netconf:
End of explanation
tm.tape
Explanation: The tape will be set accordingly:
End of explanation
reply = client.send_rpc('<run/>')
tm.state, tm.tape
Explanation: Now run the Turing Machine via RPC:
End of explanation
<END_TASK>