Remove Weird Characters using python
<p>I have this large SQL file with about 1 milllion inserts in it, some of the inserts are corrupted (about 6000) with weird characters that i need to remove so i can insert them into my DB.  </p>

<p>Ex:
INSERT INTO <code>BX-Books</code> VALUES ('2268032019','Petite histoire de la dÃƒ?Ã‚Â©sinformation','Vladimir Volkoff',1999,'Editions du Rocher','<a href="http://images.amazon.com/images/P/2268032019.01.THUMBZZZ.jpg" rel="nofollow">http://images.amazon.com/images/P/2268032019.01.THUMBZZZ.jpg</a>','<a href="http://images.amazon.com/images/P/2268032019.01.MZZZZZZZ.jpg" rel="nofollow">http://images.amazon.com/images/P/2268032019.01.MZZZZZZZ.jpg</a>','<a href="http://images.amazon.com/images/P/2268032019.01.LZZZZZZZ.jpg" rel="nofollow">http://images.amazon.com/images/P/2268032019.01.LZZZZZZZ.jpg</a>');</p>

<p>i want to remove only the weird characters and leave all of the normal ones</p>

<p>I tried using the following code to do so:</p>

<pre><code>import fileinput
import string

fileOld = open('text1.txt', 'r+')
file = open("newfile.txt", "w")

for line in fileOld: #in fileinput.input(['C:\Users\Vashista\Desktop\BX-SQL-Dump\test1.txt']):
    print(line)
    s = line
    printable = set(string.printable)
    filter(lambda x: x in printable, s)
    print(s)
    file.write(s)
</code></pre>

<p>but it doesnt seem to be working, when i print s it is the same as what is printed during line and whats stranger is that nothing gets written to the file.</p>

<p>Any advice or tips on how to solve this would be useful</p>