By default, AIX maximum password lenght is 8. This restriction is due to the default encryption algorithm.
In AIX 6.1 this can be changed (I opted out AIX 5.x since it's out of support since April 2012)

Just execute this command on the server where you need to do this:
chsec -f /etc/security/login.cfg -s usw -a pwd_algorithm=smd5

After this when you change a users password you will see a different form of hash in /etc/security/passwd.

When setting the minimum password length with "chuser" will work to put values grater than 8. (In SMIT help you will see that max password length is still mentioned 8, it's a feature, not a bug but it will work)

More information about this is on IBM website:
http://publib.boulder.ibm.com/infocenter/aix/v6r1/index.jsp?topic=%2Fcom.ibm.aix.security%2Fdoc%2Fsecurity%2Flong_passwords.htm