Glimmer of Hope!
Aha! On a hunch I go outside and check the setbacks old school: by eyeballing them in a general fashion. Sure enough looks like my two neighbors are indeed closer to the street than I am which means I am well behind the average of their two setbacks. I distinctly remember the inspector telling me that if it is obvious to the naked eye that I am farther from the street than my neighbors then I'm off the hook with the surveyor. I go inside and give the inspector a call and he tells me he'll swing by later and eyeball it himself. I could be vindicated.