Also make sure we catch all config-related errors in all related places. Fixes #324.
Closes #255. Should make a lot more sense now ;)