There are some very good climbs in the NJ Palisades, north of the GWB and south of the Tappan Zee Bridge. *Very* illegal to climb there. I mean more illegal than normal illegal
I climbed there quite a bit in the late 70's and early 80's. I climbed lines from 5.2 to 5.9-ish. Be careful of broken glass on ledges and even on small holds near overlooks. Once you get away from some of the overlooks on the Palisades Pkwy, the routes are clean aside from the occasional large, loose block. The belay ledges are cool and are mostly on top of pillars. You will be surpriased on the amount of fixed gear. There are pins and fixed nuts on the obvious lines. There is even one rad looking 'head seam [never did that one] with plenty of fixed 'heads leading the way.
We finally got caught while climbing out of one of the gullies leading to the overlook parking by a NJ State Trooper [circa spring 1986] who just happened to be pulling in [bad luck]. Put enough of a scare into us that we never went back. My climbing partner and I were just about to begin working for a defense contractor as Systems Engineers and didn't want to jepordize the jobs.