This is more an artifact of the way bound parameters are handled by the backend. Normally we would bind this to a text type as it is a string. What you can do is try setting stringtype = unspecified as a connection parameter in which case it will let the server determine the type.
PreparedStatement pstmt = connection.prepareStatement("insert into tstest values(?)");
pstmt.setObject(1, "1998-06-04 00:00:00+09");
pstmt.execute();
results in the following error:
Exception in thread "main" org.postgresql.util.PSQLException: ERROR: column "ts" is of type timestamp without time zone but expression is of type character varying
Hint: You will need to rewrite or cast the expression.
Do I understand it correctly that it is a limitation of the JDBC driver not to implement the String -> Timestamp implicit conversion listed in Table 8.5 of this document: