<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Connecting Job Titles by Topic Similarity
Given a parsed job title with known position, domain and function, we find top-$k$ titles most similar to the title.
Preps
Step1:  Load parsed titles with at least 2 posts
Step2:  Run
Step3:  Engineer Jobs
Step4:  Executive Jobs
<ASSISTANT_TASK:>
Python Code:
import gc
import ja_helpers as ja_helpers; from ja_helpers import *
gc.enable()
HOME_DIR = 'd:/larc_projects/job_analytics/'
DATA_DIR = HOME_DIR + 'data/clean/'; RES_DIR = HOME_DIR + 'results/'
domain_dir = RES_DIR + 'job_prof/by_domain/' + 'new/'
func_dir = RES_DIR + 'job_prof/by_func/' + 'new/'
Explanation: Connecting Job Titles by Topic Similarity
Given a parsed job title with known position, domain and function, we find top-$k$ titles most similar to the title.
Preps
End of explanation
title_df = pd.read_csv(DATA_DIR + 'new_titles_2posts_up.csv', keep_default_na=False)
title_df.head()
stdForm = dict(zip(title_df.non_std_title, title_df.title))
positions = dict(zip(title_df.title, title_df.position))
domains = dict(zip(title_df.title, title_df.domain))
functions = dict(zip(title_df.title, title_df.pri_func))
print('# std titles: %d' % title_df.title.nunique())
# print
Explanation: Load parsed titles with at least 2 posts: mk sure to turn off default NAs s.t empty strings will not be regarded as NAs.
End of explanation
# reload(my_util); from my_util import *
reload(ja_helpers); from ja_helpers import *
avail_domains = listFiles(domain_dir + 'sims/')
avail_funcs = listFiles(func_dir + 'sims/')
empty_sim = pd.DataFrame({'t1': [], 't2': [], 'topic_sim': []})
def getDomainSims(job_title):
    if job_title not in domains.keys():
        dom_sims = empty_sim
    else:
        dom = clean(domains[job_title])
        dom_file = domain_dir + 'sims/{}.csv'.format(dom)
        dom_sims = pd.read_csv(dom_file) if (dom in avail_domains) else empty_sim        
    return dom_sims
def getFuncSims(job_title):
    if job_title not in functions.keys():
        func_sims = empty_sim
    else:
        pri_func = clean(functions[job_title])
        func_file = func_dir + 'sims/{}.csv'.format(pri_func); 
        func_sims = pd.read_csv(func_file) if (pri_func in avail_funcs) else empty_sim
    return func_sims
## Main function 
def queryTopkSim(job_title, k=2):
    '''
    @return: k job titles with highest topic similarity to given job title
    '''
    
    job_title = camelCase(job_title)
##  Load sim scores of relevant titles (by domain or function)
    dom_sims = getDomainSims(job_title)
    func_sims = getFuncSims(job_title)
    
    # the query to retrieve all relevant obs 
    q = 't1 == "{}" or t2 == "{}"'.format(job_title, job_title)
    rel_sims = dom_sims.query(q).append(func_sims.query(q))
    
    if not rel_sims.empty: 
        return buildTopkFrom(rel_sims, k, job_title)
    else:
        return empty_sim
#         return ''
def fanOut(titles, fan_width):
    frames = [queryTopkSim(t, k=fan_width) for t in titles]
    return pd.concat(frames)
def relevantTitles(job_title='Software Developer'):
    
    job_title = camelCase(job_title)
    dom = domains[job_title] if job_title in domains.keys() else None
    func = functions[job_title] if job_title in functions.keys() else None
    if (not dom) and (not func):
        rel_titles = []
    if dom and (not func):
        rel_titles = titlesIn(dom, title_df)
    if func and (not dom):
        rel_titles = titlesWithFunc(func, title_df)
    if dom and func:
        rel_titles = np.concatenate([titlesIn(dom, title_df), titlesWithFunc(func, title_df)])
    
    return np.unique(rel_titles)
relevantTitles('software developer')
swr_dev_sims = queryTopkSim('software developer', k=5)
swr_dev_titles = swr_dev_sims.t2
fanOut(swr_dev_titles, fan_width=3)
queryTopkSim('software programmer', k=5)
Explanation: Run
End of explanation
queryTopkSim('software engineer', k=10)
queryTopkSim('mechanical engineer', k=10)
# queryTopkSim('electrical engineer', k=10)
Explanation: Engineer Jobs
End of explanation
exe_sims = pd.read_csv(func_dir + 'sims/executive.csv')
exe_sims.head()
topkByFunction(k=5, job_title='Financial Accounting Executive', func_sims=exe_sims)
exe_titles = titlesWithFunc('executive', title_df)
exe_top5 = pd.DataFrame({'title': exe_titles})
exe_top5['top5'] = exe_top5['title'].apply(topkByFunction, k=5, func_sims=exe_sims)
exe_top5.to_csv(func_dir + 'topk/executive.csv', index=False)
queryTopkSim('Accounts Executive', k=10)
queryTopkSim('Marketing Executive', k=10)
queryTopkSim('Sales Executive', k=10)
sales_exe_top10 = queryTopkSim('Sales Executive', k=10)
fan = fanOut(sales_exe_top10.t2, fan_width=2)
cand_titles = fan.t2
cand_titles
# TESTS
# swr_sims = pasteCols('t2', 'topic_sim', swr_sims, 'title_n_sim')
# swr_sims.head()
# swapCols('t1', 't2', swr_sims.head())
Explanation: Executive Jobs
End of explanation
<END_TASK>