<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Exercise 02
Step1:  The variable x is a string in Python
Step2:  Its translation into ASCII is unusable by parsers
Step3:  Encoding as UTF-8 doesn't help much - use it as an encoder
Step4:  Ignoring difficult characters is perhaps an even worse strategy - ignore everything that is not understood
Step5:  However, one can normalize then encode…
Step6:  Even before this normalization and encoding, you may need to convert some characters explicitly before parsing. For example
Step7:  Then consider the results here
Step8:  One of the ways to handle punctuations...which drops characters that may be important for parsing a sentence, so instead
<ASSISTANT_TASK:>
Python Code:
x = "Rinôçérôse screams ﬂow not unlike an encyclopædia, \
'TECHNICIÄNS ÖF SPÅCE SHIP EÅRTH THIS IS YÖÜR CÄPTÅIN SPEÄKING YÖÜR ØÅPTÅIN IS DEA̋D' to Spın̈al Tap."
type(x)
Explanation: Exercise 02: concerns about characters
The following shows examples of how to use codecs and normalize unicode, and draws heavily from the article Metal umlat.
End of explanation
repr(x)
Explanation: The variable x is a string in Python:
End of explanation
ascii(x)
Explanation: Its translation into ASCII is unusable by parsers:
End of explanation
x.encode('utf8')
Explanation: Encoding as UTF-8 doesn't help much - use it as an encoder:
End of explanation
x.encode('ascii','ignore')
Explanation: Ignoring difficult characters is perhaps an even worse strategy - ignore everything that is not understood:
End of explanation
import unicodedata
# NFKD a robust way to handle normalizers - convert special characters into something
# that can be read and convert into ascii
unicodedata.normalize('NFKD', x).encode('ascii','ignore')
Explanation: However, one can normalize then encode…
End of explanation
x = "The sky “above” the port … was the color of ‘cable television’ – tuned to the Weather Channel®"
ascii(x)
Explanation: Even before this normalization and encoding, you may need to convert some characters explicitly before parsing. For example:
End of explanation
unicodedata.normalize('NFKD', x).encode('ascii','ignore')
Explanation: Then consider the results here:
End of explanation
x = x.replace('“', '"').replace('”', '"')
x = x.replace("‘", "'").replace("’", "'")
x = x.replace('…', '...').replace('–', '-')
print(x)
Explanation: One of the ways to handle punctuations...which drops characters that may be important for parsing a sentence, so instead:
End of explanation
<END_TASK>