<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Define the task
In this example our task is to make a model which would distinguish quasars from Be stars which light curves are very simmilar.
Step1:  Show available descriptors and classifiers
Note
Step2:  Note
Step3:  Make list of all parameters combination
Step4:  Load stars
Step5:  Clean sample
Keep just stars with light curves which have certain size
Step6:  Train - test split
Step7:  Show random quasars
Step8:  Plot distribution of magnitudes, variogram and light curve of random quasars
Step9:  Find optimal parameters
Step10:  Show features for some quasars
Step11:  Evaluate model
Now we can use tuned filter to predict whether given stars are quasars or not. Note that we're evaluating on the test
sample.
Step12:  Probability plot
Purple stars are quasars, black are Be stars. Color on the background represents probability of an object
to be quasar (scaled by colorbar on the right)
Step13:  Find a quasar candidates in OGLEII
NOTE
Step14:  Redis searcher
<ASSISTANT_TASK:>
Python Code:
# We will be using Abbe value and variogram slope to describe each light curve
# and the "brain" of the filter will be Gradient Boosting
descr_name1 = "AbbeValueDescr"
descr_name2 = "VariogramSlopeDescr"
decid_name = "GradBoostDec"
# Parameters to tune and their ranges
tun_param1 = "bins"
bin_from = 10
bin_to = 150
bin_step = 50
tun_param2 = "days_per_bin"
dpb_from = 30
dpb_to = 110
dpb_step = 10
# Load example stars which are included in the package
obt_method = "FileManager"
quasars_path = PackageReader.getSamplePath("qso")
be_stars_path = PackageReader.getSamplePath("be_stars")
Explanation: Define the task
In this example our task is to make a model which would distinguish quasars from Be stars which light curves are very simmilar.
End of explanation
all_descriptors = PackageReader().getClassesDict("descriptors")
all_deciders = PackageReader().getClassesDict("deciders")
print("Descriptors: {}\n".format(", ".join(all_descriptors.keys())))
print("Deciders: {}".format(", ".join(all_deciders.keys())))
Explanation: Show available descriptors and classifiers
Note: You can arbitrary number of descriptors and classifiers
End of explanation
abbe_descr = all_descriptors.get(descr_name1)
vario_slope_descr = all_descriptors.get(descr_name2)
decider = all_deciders.get(decid_name)
Explanation: Note: all_descriptors and all_deciders are dictionaries of descriptors and deciders classes which can be directly used. For example:
my_descriptor = all_descriptors["AbbeValueDescr"](bins=100)
is same as:
```
from lcc.stars_processing.descriptors import AbbeValueDescr
my_descriptor = AbbeValueDescr(bins=100)
```
Get descriptors and a classifier
End of explanation
combinations = get_combinations([":".join([descr_name1, tun_param1]), ":".join([descr_name2, tun_param2])],
                                 range(bin_from, bin_to, bin_step), range(dpb_from, dpb_to, dpb_step))
tun_params = parse_tun_query(combinations)
tun_params[:3]
Explanation: Make list of all parameters combination
End of explanation
quasars = FileManager({"path": quasars_path}).getStars()
be_stars = FileManager({"path": be_stars_path}).getStars()
Explanation: Load stars
End of explanation
quasars = [st for st in quasars if st.lightCurve and len(st.lightCurve.mag) >= bin_to]
be_stars = [st for st in be_stars if st.lightCurve and len(st.lightCurve.mag) >= bin_to]
Explanation: Clean sample
Keep just stars with light curves which have certain size
End of explanation
quasars_train, quasars_test = train_test_split(quasars, train_size=0.8)
be_stars_train, be_stars_test = train_test_split(be_stars, train_size=0.8)
Explanation: Train - test split
End of explanation
import random
random.shuffle(quasars)
for star in quasars[:3]:
    print("*"*20)
    print(star)
    print()
Explanation: Show random quasars
End of explanation
plotStarsPicture(quasars[:3])
# Light curves description
quasars[0].lightCurve.meta
Explanation: Plot distribution of magnitudes, variogram and light curve of random quasars
End of explanation
# Estimate all combinations and get the best one
es = ParamsEstimator(searched=quasars_train,
                     others=be_stars_train,
                     descriptors=[abbe_descr, vario_slope_descr],
                     deciders=[decider],
                     tuned_params=tun_params)
star_filter, best_stats, best_params = es.fit()
print("Optimal parameters: {}".format(es.tuned_params[0]))
pd.DataFrame(es.stats)
Explanation: Find optimal parameters
End of explanation
star_filter.getSpaceCoordinates(quasars[:5])
Explanation: Show features for some quasars
End of explanation
prediction_quasars = star_filter.evaluateStars(quasars_test)
prediction_be_stars = star_filter.evaluateStars(be_stars_test)
prediction_quasars.head()
Explanation: Evaluate model
Now we can use tuned filter to predict whether given stars are quasars or not. Note that we're evaluating on the test
sample.
End of explanation
plotProbabSpace(star_filter)
Explanation: Probability plot
Purple stars are quasars, black are Be stars. Color on the background represents probability of an object
to be quasar (scaled by colorbar on the right)
End of explanation
# This should return a star from OGLEII
ogle_star = OgleII({"starid": 1, "field_num": 2, "target": "lmc"}).getStars()
# Query for OgleII
db_name = "OgleII"
starid_from = 1
starid_to = 10
field_num_from = 1
field_num_to = 2
target = "lmc"
# Prepare queries and run sequential systematic search by using filter
queries = get_combinations(["starid", "field_num", "target"],
                           range(starid_from, starid_to),
                           range(field_num_from, field_num_to),
                           [target])
searcher = StarsSearcher([star_filter],
                         db_connector=db_name,
                         stat_file_path="/tmp/lcc_status.csv",
                        save_coords=True)
searcher.queryStars(queries)
passed_stars = searcher.getPassedStars()
searcher.getStatus()
saveStars(quasars[:5], "/tmp")
Explanation: Find a quasar candidates in OGLEII
NOTE: Sometimes OgleII doesn't work because their database, so if the query takes too long, maybe it's that case
End of explanation
redis_searcher = StarsSearcherRedis([star_filter], db_connector=db_name, save_coords=True)
redis_searcher.queryStars(queries)
passed_stars = redis_searcher.getPassedStars()
# redis_searcher.getStatus()
Explanation: Redis searcher
End of explanation
<END_TASK>