Worked in New York off and on over a six month period in 1995. The company put me up in Newark. I believe it was a Hilton, but it is connected to Newark Penn Station by a covered walkway. From Newark Penn Station you can easily get to most places on Manhattan. New Jersey Transit to Penn Station or PATH trains to the World Trade Center site, and then get on the New York Subway to anywhere. Costs of things in New York City are high. Expect them to be roughly double Atlanta.
Don't be stupid in where you go when. You will not blend in, even if you think you do. In fact, you will be carrying a sign invisible only to you and other tourists that says "Out of town tourist" On the other hand, a lot of the horror stories are exaggerated. Generally mornings are better tourist times, as the wierdos tend to sleep late.
Get a subway system map. A bus system map is also available, but unless they are changed, the busses in Atlanta are much nicer than in New York. Do not let Atlanta MARTA define your expectations for the New York subway. For the most part it is a very old system. In may locations the signage is neither obvious nor readily understandable to the outsider. The ridership is like a visit to a homan zoo, particularly outside of rush hour. If you want more, email me.
There are many interesting things to see and do, but I would never want to live there.
I left to take another overseas job, becuase, as a southerner I feel less in foreign territory in parts of Asia than in New York.