The COBCH0801 was caused by a syntax error in my SQL statement. I fixed this and the compile error went away.
I am running esqlconfigw.exe as administrator and I'll open a support incident on the error I'm getting.
I am setting the SQL(BEHAVIOR=UNOPTIMIZED) directive in my project, in case that's needed.
The format 6 CONNECT returns error -19703, could not connect.
My bind connection looks like this, where sql-conn and sql-trans are defined in the SQL DECLARE section as type object. I am setting sql-conn to an instance of the Postgresql data handler's connection class and sql-trans to null:
EXEC ADO
BIND CONNECTION TO :sql-conn WITH TRANSACTION :sql-trans
END-EXEC
I am able to connect successfully, but EXEC ADO BIND CONNECTION returns error code -2146233080, index was outside the bounds of the array.
I can still access my database using the classes in the data handler. I just can't do embedded SQL.