<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Data Input Tables and Sources
Module to simplify handling of input and output tables (as .csv text and/or files).
There are two main classes defined here
Step2:  class Table
Step3:  class DataSource
Class to unify the source of tables.  For now, this assumes that a table
Step4:  Reading Tables
Step5:  Writing Tables
<ASSISTANT_TASK:>
Python Code:
from salib import extend
import pandas as pd
import os, os.path
try:
    from StringIO import StringIO
except ImportError:
    from io import StringIO
import hashlib
from IPython.core.magic import register_cell_magic
import re
Explanation: Data Input Tables and Sources
Module to simplify handling of input and output tables (as .csv text and/or files).
There are two main classes defined here:
class Table - this is a slightly augmented subclass of a pandas 
DataFrame; augmented
to retain the name of the table and the source of the data.
class DataSource - this class manages the various possible data sources to provide a unified
API for programs that need the data.  The most important method, '.read_table()', returns a single
named Table.  Currently, it tries the following sources in order, returning the first
one it finds:
a table provided directly by an application via the '.set_table()' method.
a table provided as CSV data directly in a cell using the '%%Table' cell magic.
a table in a CSV file with path name: 'root/dsname.d/tablename.csv'
root is the root directory of all data sets, default 'data'.
dsname is a data set name specified when an analysis is started by instantiating Frame2D.
tablename is the name of the table.
End of explanation
class Table(pd.DataFrame):
    
    A Table is just like a pandas DataFrame except that it has
    a table name, a data set name, and a file name - the latter two describing
    the source of the data.
    
    _internal_names = pd.DataFrame._internal_names + ['filename','tablename']
    _internal_names_set = set(_internal_names)
    _metadata = ['dsname']
            
    def __init__(self,*args,**kwargs):
        dsname = kwargs.pop('dsname',None)
        tablename = kwargs.pop('tablename',None)
        filename = kwargs.pop('filename',None)
        super(self.__class__,self).__init__(*args,**kwargs)
        if dsname is not None:
            self.dsname = dsname
        if tablename is not None:
            self.tablename = tablename
        if filename is not None:
            self.filename = filename
        
    @property
    def _constructor(self):
        return self.__class__
##test:
t = Table(data=[(10,20.,'a'),(11,22.,'b'),(12,23.,'c')],
          columns=['I','F','S'],tablename='Test',dsname='Notebook')
t
##test:
t.dtypes
##test:
t.tablename, t.dsname
##test:
t2 = t[['S','I']]
t2
##test:
hasattr(t2,'tablename'), hasattr(t2,'dsname')
##test:
t2.dsname
##test:
t = pd.DataFrame(data=[(10,20.,'a'),(11,22.,'b'),(12,23.,'c')],columns=['I','F','S'])
u = Table(data=t,dsname='foo',copy=False)
u
##test:
u['F'] *= 3
u
##test:
t
##test:
u.dsname
Explanation: class Table
End of explanation
class DataSource(object):
    
    ROOT = 'data'
    DSNAME = None     # default data set name
    DSTYPE = 'dir'    # someday we will allow 'zip' for zip archives
    #DSTYPE = 'cell'  # for CSV data provided via %%Table cell magic
    #DSTYPE = 'data'  # for dataframe data provided directly
    CELLDATA = {}     # csv text from %%Table magic cells, indexed by table name
    TABLES = {}       # dataframes directly provided by client, indexed by table name
    
    DATASOURCE = None # the one and only data source
    
    def __init__(self):
        cls = self.__class__
        if cls.DATASOURCE is not None:
            raise ValueError("Can only create one instance of class '{}'".format(cls.__name__))
        self.root = cls.ROOT
        self.dsname = cls.DSNAME
        self.prefix = None
        self.dstype = cls.DSTYPE
        self.celldata = cls.CELLDATA
        self.tables = cls.TABLES
        cls.DATASOURCE = self
##test:
d = DataSource()
vars(d)
##test:
try:
    d2 = DataSource()
except Exception as e:
    print('*'*5,e)
    d2 = None
d2
@extend
class DataSource:
    
    @classmethod
    def set_root(cls,newroot):
        self = cls.DATASOURCE
        if not os.path.exists(newroot):
            raise ValueError("Root '{}' does not exist.".format(newroot))
        self.root = newroot
    @classmethod
    def set_source(cls,dsname,dstype=None):
        self = cls.DATASOURCE
        if dsname is not None:
            if dstype is None:
                dirname = self.root + '/' + dsname + '.d'
                if os.path.exists(dirname):
                    dstype = 'dir'
                else:
                    dstype = 'unknown'
            if dstype not in ['dir','cell','data']:
                raise ValueError("dstype '{}' is invalid.".format(dstype))
        self.dsname = dsname
        self.dstype = dstype
        self.celldata = {}
        self.tables = {}
        
    @classmethod
    def set_table(cls,tablename,table):
        self = cls.DATASOURCE
        self.tables[tablename] = table
        if tablename in self.celldata:
            del self.celldata[tablename]
    
    @classmethod
    def set_celldata(cls,tablename,celltext):
        self = cls.DATASOURCE
        self.celldata[tablename] = celltext
        if tablename in self.tables:
            del self.tables[tablename]
    
    def _file_name(self,tablename,prefix=None):
        n = tablename
        if prefix:
            n = prefix + '/' + tablename
        return self.root + '/' + self.dsname + '.d/' + n + '.csv'
##test:
DataSource.DATASOURCE = None
ds = DataSource()
vars(ds)
##test:
try:
    DataSource.set_root('foo')
except Exception as e:
    print('*'*5,e)
vars(ds)
##test:
DataSource.set_root('img')
vars(ds)
##test:
DataSource.set_root('data')
##test:
DataSource.set_source('frame-1')
vars(ds)
##test:
DataSource.set_table('joints',[dict(NODEID='A',X=10,Y=20),dict(NODEID='B',Y=20,X=30)])
vars(ds)
##test:
DataSource.set_celldata('joints','NODEID,X,Y\nA,10,20\nB,30,20')
vars(ds)
##test:
ds._file_name('joints')
##test:
ds._file_name('joints',prefix='lcase1')
Explanation: class DataSource
Class to unify the source of tables.  For now, this assumes that a table:
 * has been specified directly via '.set_table', or
 * has been provided in CSV form using the cell magic '%%Table', or
 * is in a CSV file available in a directory '&lt;root&gt;/xyz.d',  where 'xyz' is the 
   'data set name'.  
Eventually, we will have a way of archiving sets of files in .zip files.
End of explanation
@extend
class DataSource:
    
    @classmethod
    def read_table(cls,tablename,optional=False,prefix=None,columns=None,extrasok=True):
        self = cls.DATASOURCE
        stream = None
        filename = None
        t = None
        
        def _chk(t,columns=columns):
            if columns is None:
                return t
            prov = set(t.columns)
            reqd = set(columns)
            if reqd-prov:
                raise ValueError("Columns missing for table '{}': {}. Required columns are: {}"
                                 .format(tablename,list(reqd-prov),columns))
            if prov-reqd:
                if not extrasok:
                    raise ValueError("Extra columns for table '{}': {}. Required columns are: '{}'"
                                    .format(tablename,list(prov-reqd),columns))
                t = t[columns]
            return t            
            
        if tablename in self.tables:
            return _chk(self.tables[tablename])
        if tablename in self.celldata:
            stream = StringIO(self.celldata[tablename])
        else:
            if self.dsname is not None:
                filename = self._file_name(tablename,prefix=prefix)
                if os.path.exists(filename):
                    stream = open(filename,'r')
        if stream is None:
            if optional:
                d = pd.DataFrame(columns=columns)
            else:
                raise ValueError("Table '{}' does not exist.".format(tablename))
        else:
            d = pd.read_csv(stream,index_col=None,skipinitialspace=True)
        t = Table(d,dsname=self.dsname,tablename=tablename,filename=filename)
        return _chk(t)
##test:
DataSource.set_source('frame-6')
t = DataSource.read_table('nodes')
t
##test:
type(t)
##test:
len(t)
##test:
t[['X','Y']] /= 3.
t
##test:
vars(t)
##test:
DataSource.read_table('nodes',columns=['NODEID','Y','X'])
##test:
try:
    t = DataSource.read_table('nodes',columns=['NODEID','Y','X'],extrasok=False)
except Exception as e:
    print('***',e)
    t = None
t
##test:
try:
    t = DataSource.read_table('nodes',columns=['NODEID','Y','X','C','D'])
except Exception as e:
    print('***',e)
    t = None
t
##test:
try:
    t = DataSource.read_table('nodesxxx',columns=['NODEID','Y','X'],extrasok=False)
except Exception as e:
    print('***',e)
    t = None
t
##test:
try:
    t = DataSource.read_table('nodesxxx',columns=['NODEID','Y','X'],extrasok=False,optional=True)
except Exception as e:
    print('***',e)
    t = None
t
@register_cell_magic('Table')
def cell_table(line,celltext):
    mo = re.match(r'\s*(\S+)\s*$',line)
    if not mo:
        raise ValueError('Usage: %%Table tablename')
    tablename = mo.group(1)
    global DataSource
    DataSource.set_celldata(tablename,celltext)
%%Table nodes
NODEID,X,Y,Z
A,0.,0.,50001
B,0,4000,50002
C,8000,4000,50003
D,8000,0,50004
##test:
t2 = DataSource.read_table('nodes')
t2
##test:
DataSource.set_table('nodes',t2+t2)
t3 = DataSource.read_table('nodes')
t3
##test:
vars(t2)
Explanation: Reading Tables
End of explanation
@extend
class DataSource:
    
    @classmethod
    def write_table(cls,table,root=None,dsname=None,tablename=None,prefix=None,precision=None,index=False,makedir=False):
        self = cls.DATASOURCE
        if root is None:
            root = self.root
        if dsname is None:
            dsname = self.dsname
        if tablename is None:
            tablename = table.tablename
        dirname = root + '/' + dsname + '.d'
        if makedir and not os.path.exists(dirname):
            os.mkdir(dirname)
        if prefix is not None:
            dirname = dirname + '/' + prefix
            if makedir and not os.path.exists(dirname):
                os.mkdir(dirname)
                
        table.tablename = tablename
        table.dsname = dsname
        table.filename = filename = dirname + '/' + tablename + '.csv'
        
        float_format = None
        if precision is not None:
            float_format = '%.{:d}g'.format(precision)
        table.to_csv(filename,index=index,float_format=float_format)
        return filename
@extend
class Table:
    
    def signature(self):
        filename = self.filename
        if os.path.exists(filename):
            return (self.tablename,self.filename,signature(filename))
        raise ValueError("Table {}: filename: {} - does not exist.".format(self.tablename,self.filename))
    
def signature(filename):
    f = open(filename,mode='rb')
    m = hashlib.sha256(f.read())
    f.close()
    return m.hexdigest()
DataSource.DATASOURCE = None
__ds__ = DataSource()
%%Table nodes
NODEID,X,Y,Z
A,0.,0.,6002.
B,0,4000,7003
C,8000,4000,8004
D,8000,0,9005
##test:
t = DataSource.read_table('nodes')
t
##test:
t[['X','Z']] /= 3
t
##test:
vars(t)
##test:
try:
    DataSource.write_table(t,dsname='test',prefix='pfx',tablename='nodes2')
except Exception as e:
    print('*'*5,e)
##test:
%rm -rf data/test.d
try:
    r = DataSource.write_table(t,dsname='test',prefix='pfx',tablename='nodes2',makedir=True,precision=15)
except Exception as e:
    print('*'*5,e)
r
##test:
%cat data/test.d/pfx/nodes2.csv
##test:
t.signature()
##test:
%rm -rf data/test.d
##test:
vars(t)
DataSource.DATASOURCE = None
__ds__ = DataSource()
Explanation: Writing Tables
End of explanation
<END_TASK>