IANAL. But I have done this a number of times, though not specifically in NYC.
Yes, you will need to register as a foreign LLC in your home state if you transact business from that state. Each state has specific rules on what "transacting business" means, but generally if you work from that state or have a bank account that you routinely access or have mail delivered there you will be required to register there. My state has some ways around it but almost every CPA/Tax person I have talked to always suggests the foreign registration even when it may not be 100% required just because of the potential for liabilities adding up. So you will owe NYC LLC taxes in addition to normal Federal taxes and Delaware annual LLC fees.
My 2 cents, there is no reason to register in Delaware as an LLC and then register in your home state. Just register in your home state and save yourself some money and headaches. While there are some advantages of a Delaware LLC, there is also a number of extra fees, steps and caveats. So unless you have a reason to need those specific benefits and are willing to pay advisors to make sure you don't have (or accidentally create) issues it is easier and less complicated to just start with your home state.
One advantage of a Delaware LLC people often cite is the personal liability protection which is better than many other states, however, since your primary business is in NYC and operating under the laws of NYC it becomes a little more complex than that simple statement.