You may not have the needed permissions to create the AdminStudio catalog database.
It's easiest to provide AdminStudio with a SQL account with the rights to create the database but if your DBA won't allow that, you can use .sql scripts to manually create the database.
The process of manually creating the catalog database is described on pg 44 of the "Workflow Manager 6.0 / AdminStudio Enterprise Server 11.5 Installation Guide"
http://www.flexerasoftware.com/webdocuments/PDF/AS115-WFM6-InstallGuide.pdfThe gist of it is:
Open SQL Server Management Studio and create a new database. The database collation must be SQL_Latin1_General_CP1_CI_AS
The .sql scripts are located at [ProgramFilesFolder]AdminStudio\xx.x\Support\SQL_Scripts\
Execute the following .sql scripts in this order:
01. AMS_System_Schema.SQL
02. WFM_System_Data.SQL
03. AMSCreateIndex.SQL
04. WFM_SampleTemplates.SQL
05. AS_System_Schema.SQL
06. Seed_Data.SQL
07. CustomReportWizard.SQL
08. Reporting.StoredProcedures.SQL
The Windows or SQL account used to access AdminStudio requires db_owner rights on the database. This is because of the need for rights to create new tables in the database (for activities such as importing custom tables from .MSI's when bringing them into the Application Manager). db_datareader and db_datawriter is not good enough (it's gotta be db_owner).
**Warning! I think there is a serious problem with manually creating the catalog database for AdminStudio 11.5. The above directions work for AdminStudio 11.0. My recommendation (for people who must manually create the database) is to install AdminStudio 11.0 and manually create the database and then install AdminStudio 11.5 and allow it to upgrade your database.
I have an incident logged with support and will update this post when I get further info. If you must manually create the AdminStudio 11.5 catalog database, please PM or e-mail me for more info. I have been successful in manually creating the 11.5 catalog database, but it's a bit involved.