Clean and Scrape Before You Paint
It seems counterintuitive to clean the windows first. But it's a good idea for two reasons. First, your paint job will last longer if the intersection of the glass and wood sash is sealed with paint. You can't get a good seal if the glass is dirty. Second, you'll get a neater paint job if you scrape off previous layers of paint and other crud that may be on the glass.
Use any window washing solution you prefer and a new, single-edge blade to clean old paint and gunk from the glass. Apply the solution, then scrape, to avoid scraping on dry glass. Never scrape dry glass. Grit pushed by the razor can scratch the window.