You can get a French threaded headset here -
http://www.ebay.co.uk/itm/200650061768?ssPageName=STRK:MEWAX:IT&_trksid=p3984.m1423.l2649 - I bought one for my old Peugeot touring bike. Seems to be decent quality for the money.
I'd imagine you should also be able to source French (or it could be Swiss!) BB cups from France as well. If it's cottered cranks, you will need 9mm cotter pins (everybody else used 3/8"). If it's square tapered or you want to convert to square taper, probably the easiest thing is to buy a Prestine threadless BB from Parkers of Bolton. It will fit.
Also be aware that the French stem is 22m whereas everything else is 7/8" In reality, the difference is minute so a standard stem will possibly fit anyway or it will fit with a little sandpapering.
My biggest concern assuming the wheels are serviceable is that the French freewheel thread is different from the norm so if you want to use the original back wheel but change the gearing, you may have a bit of difficulty tracking down a freewheel. Your choice would certainly be limited. However, a budget wheel with a Quando hub will work perfectly well and these are better hubs than people make out, the secret being to grease and adjust the bearing properly before use.
Also, make sure you have the original seatpost as the French used some odd sizes.