So "the flipper" is not acknowledging your complaint?
Call the City permits office and/or The Ministry of Labour, or at least use the threat of a call to gain his attention.
His 'Building Permit' and a 'Notice of Project' must be conspicuously posted (usually a window or on the front door).
If it's not there, implying he may be doing this work without a permit, they can and will halt construction immediately and indefinitely.
The last thing any 'flipper' wants is the aggravation of a time delay.
As a side note, document everything, including your efforts to reach an amicable settlement, names on disposal bins and pick-ups, photos of workers, etc..
Don't let it ride - guys like this give the construction industry a bad name.