How can I keep from having to use srm.conf and access.conf?

You can override these two files by putting ResourceConfig and/or AccessConfig directives in your httpd.conf. If you do that the other two files will not be read. Notably the default install of SWS comes preconfigured to only use the httpd.conf file. If it's using the srm.conf and access.conf files you have made changes to the defaults which enabled those files.