OS X: launchctl: error unloading

Need to uninstall xquartz but keep running into the following error?

macbook:~ clem$ sudo launchctl unload -w /Library/LaunchAgents/org.macosforge.xquartz.startx.plist launchctl: Error unloading: org.macosforge.xquartz.startx
For whatever reason, you have to run this particular unload as a regular user and not as root or using sudo. Once you do that, it should successfully unload.