Cannot create Java VM on OpenVZ

I'm constantly encountering an error related to Java and certificates on my Ubuntu server running in OpenVZ when installing things from apt-get. I'm pretty sure it has to do with how Java allocates memory. I know the fail counter for privvmpages is very high, so the problem must be that Java is hitting this limit.

I have read that the server VM will allocate a lot of memory up front to preempt performance issues, but that the client VM doesn't do this and might be better for what I'm doing. I messed with jvm.cfg to make the system go to the client VM, but get an error that it can't find the client VM.

I have tried replacing the Java binary with a script calling Java with -Xms and -Xmx settings, and that solves the issue for when I call basic things from the command line, but not for when doing things like having apt-get configure certificates.

I'm at a loss for what to try next. I need to get this working, but simply increasing privvmpages is not an available option. I have the actual error pasted below.

Setting up ca-certificates-java (20100412) ...
creating /etc/ssl/certs/java/cacerts...
Could not create the Java virtual machine.
  error adding brasil.gov.br/brasil.gov.br.crt
  error adding cacert.org/cacert.org.crt
  error adding debconf.org/ca.crt
  error adding gouv.fr/cert_igca_dsa.crt
  error adding gouv.fr/cert_igca_rsa.crt
  error adding mozilla/ABAecom_=sub.__Am._Bankers_Assn.=_Root_CA.crt
  error adding mozilla/AOL_Time_Warner_Root_Certification_Authority_1.crt
  error adding mozilla/AOL_Time_Warner_Root_Certification_Authority_2.crt
  error adding mozilla/AddTrust_External_Root.crt
  error adding mozilla/AddTrust_Low-Value_Services_Root.crt
  error adding mozilla/AddTrust_Public_Services_Root.crt
  error adding mozilla/AddTrust_Qualified_Certificates_Root.crt
  error adding mozilla/America_Online_Root_Certification_Authority_1.crt
  error adding mozilla/America_Online_Root_Certification_Authority_2.crt
  error adding mozilla/Baltimore_CyberTrust_Root.crt
  error adding mozilla/COMODO_Certification_Authority.crt
  error adding mozilla/COMODO_ECC_Certification_Authority.crt
  error adding mozilla/Camerfirma_Chambers_of_Commerce_Root.crt
  error adding mozilla/Camerfirma_Global_Chambersign_Root.crt
  error adding mozilla/Certplus_Class_2_Primary_CA.crt
  error adding mozilla/Certum_Root_CA.crt
  error adding mozilla/Comodo_AAA_Services_root.crt
  error adding mozilla/Comodo_Secure_Services_root.crt
  error adding mozilla/Comodo_Trusted_Services_root.crt
  error adding mozilla/DST_ACES_CA_X6.crt
  error adding mozilla/DST_Root_CA_X3.crt
  error adding mozilla/DigiCert_Assured_ID_Root_CA.crt
  error adding mozilla/DigiCert_Global_Root_CA.crt
  error adding mozilla/DigiCert_High_Assurance_EV_Root_CA.crt
Could not create the Java virtual machine.
  error adding mozilla/Digital_Signature_Trust_Co._Global_CA_1.crt
  error adding mozilla/Digital_Signature_Trust_Co._Global_CA_2.crt
  error adding mozilla/Digital_Signature_Trust_Co._Global_CA_3.crt
  error adding mozilla/Digital_Signature_Trust_Co._Global_CA_4.crt
  error adding mozilla/Entrust.net_Global_Secure_Personal_CA.crt
  error adding mozilla/Entrust.net_Global_Secure_Server_CA.crt
  error adding mozilla/Entrust.net_Premium_2048_Secure_Server_CA.crt
  error adding mozilla/Entrust.net_Secure_Personal_CA.crt
  error adding mozilla/Entrust.net_Secure_Server_CA.crt
  error adding mozilla/Entrust_Root_Certification_Authority.crt
  error adding mozilla/Equifax_Secure_CA.crt
  error adding mozilla/Equifax_Secure_Global_eBusiness_CA.crt
  error adding mozilla/Equifax_Secure_eBusiness_CA_1.crt
  error adding mozilla/Equifax_Secure_eBusiness_CA_2.crt
  error adding mozilla/Firmaprofesional_Root_CA.crt
  error adding mozilla/GTE_CyberTrust_Global_Root.crt
  error adding mozilla/GTE_CyberTrust_Root_CA.crt
  error adding mozilla/GeoTrust_Global_CA.crt
  error adding mozilla/GeoTrust_Global_CA_2.crt
  error adding mozilla/GeoTrust_Primary_Certification_Authority.crt
  error adding mozilla/GeoTrust_Universal_CA.crt
  error adding mozilla/GeoTrust_Universal_CA_2.crt
  error adding mozilla/GlobalSign_Root_CA.crt
  error adding mozilla/GlobalSign_Root_CA_-_R2.crt
  error adding mozilla/Go_Daddy_Class_2_CA.crt
  error adding mozilla/IPS_CLASE1_root.crt
  error adding mozilla/IPS_CLASE3_root.crt
  error adding mozilla/IPS_CLASEA1_root.crt
  error adding mozilla/IPS_CLASEA3_root.crt
  error adding mozilla/IPS_Chained_CAs_root.crt
  error adding mozilla/IPS_Servidores_root.crt
  error adding mozilla/IPS_Timestamping_root.crt
  error adding mozilla/NetLock_Business_=Class_B=_Root.crt
  error adding mozilla/NetLock_Express_=Class_C=_Root.crt
  error adding mozilla/NetLock_Notary_=Class_A=_Root.crt
  error adding mozilla/NetLock_Qualified_=Class_QA=_Root.crt
  error adding mozilla/Network_Solutions_Certificate_Authority.crt
  error adding mozilla/QuoVadis_Root_CA.crt
  error adding mozilla/QuoVadis_Root_CA_2.crt
  error adding mozilla/QuoVadis_Root_CA_3.crt
  error adding mozilla/RSA_Root_Certificate_1.crt
  error adding mozilla/RSA_Security_1024_v3.crt
  error adding mozilla/RSA_Security_2048_v3.crt
  error adding mozilla/SecureTrust_CA.crt
  error adding mozilla/Secure_Global_CA.crt
  error adding mozilla/Security_Communication_Root_CA.crt
  error adding mozilla/Sonera_Class_1_Root_CA.crt
  error adding mozilla/Sonera_Class_2_Root_CA.crt
  error adding mozilla/Staat_der_Nederlanden_Root_CA.crt
  error adding mozilla/Starfield_Class_2_CA.crt
  error adding mozilla/StartCom_Certification_Authority.crt
  error adding mozilla/StartCom_Ltd..crt
  error adding mozilla/SwissSign_Gold_CA_-_G2.crt
  error adding mozilla/SwissSign_Platinum_CA_-_G2.crt
  error adding mozilla/SwissSign_Silver_CA_-_G2.crt
  error adding mozilla/Swisscom_Root_CA_1.crt
  error adding mozilla/TC_TrustCenter__Germany__Class_2_CA.crt
  error adding mozilla/TC_TrustCenter__Germany__Class_3_CA.crt
  error adding mozilla/TDC_Internet_Root_CA.crt
  error adding mozilla/TDC_OCES_Root_CA.crt
  error adding mozilla/TURKTRUST_Certificate_Services_Provider_Root_1.crt
  error adding mozilla/TURKTRUST_Certificate_Services_Provider_Root_2.crt
  error adding mozilla/Taiwan_GRCA.crt
  error adding mozilla/Thawte_Personal_Basic_CA.crt
  error adding mozilla/Thawte_Personal_Freemail_CA.crt
  error adding mozilla/Thawte_Personal_Premium_CA.crt
  error adding mozilla/Thawte_Premium_Server_CA.crt
  error adding mozilla/Thawte_Server_CA.crt
  error adding mozilla/Thawte_Time_Stamping_CA.crt
  error adding mozilla/UTN-USER_First-Network_Applications.crt
  error adding mozilla/UTN_DATACorp_SGC_Root_CA.crt
  error adding mozilla/UTN_USERFirst_Email_Root_CA.crt
  error adding mozilla/UTN_USERFirst_Hardware_Root_CA.crt
  error adding mozilla/ValiCert_Class_1_VA.crt
  error adding mozilla/ValiCert_Class_2_VA.crt
  error adding mozilla/VeriSign_Class_3_Public_Primary_Certification_Authority_-_G5.crt
  error adding mozilla/Verisign_Class_1_Public_Primary_Certification_Authority.crt
  error adding mozilla/Verisign_Class_1_Public_Primary_Certification_Authority_-_G2.crt
  error adding mozilla/Verisign_Class_1_Public_Primary_Certification_Authority_-_G3.crt
  error adding mozilla/Verisign_Class_2_Public_Primary_Certification_Authority.crt
  error adding mozilla/Verisign_Class_2_Public_Primary_Certification_Authority_-_G2.crt
  error adding mozilla/Verisign_Class_2_Public_Primary_Certification_Authority_-_G3.crt
  error adding mozilla/Verisign_Class_3_Public_Primary_Certification_Authority.crt
  error adding mozilla/Verisign_Class_3_Public_Primary_Certification_Authority_-_G2.crt
  error adding mozilla/Verisign_Class_3_Public_Primary_Certification_Authority_-_G3.crt
  error adding mozilla/Verisign_Class_4_Public_Primary_Certification_Authority_-_G2.crt
  error adding mozilla/Verisign_Class_4_Public_Primary_Certification_Authority_-_G3.crt
  error adding mozilla/Verisign_RSA_Secure_Server_CA.crt
  error adding mozilla/Verisign_Time_Stamping_Authority_CA.crt
  error adding mozilla/Visa_International_Global_Root_2.crt
  error adding mozilla/Visa_eCommerce_Root.crt
  error adding mozilla/WellsSecure_Public_Root_Certificate_Authority.crt
  error adding mozilla/Wells_Fargo_Root_CA.crt
  error adding mozilla/XRamp_Global_CA_Root.crt
  error adding mozilla/beTRUSTed_Root_CA-Baltimore_Implementation.crt
  error adding mozilla/beTRUSTed_Root_CA.crt
  error adding mozilla/beTRUSTed_Root_CA_-_Entrust_Implementation.crt
  error adding mozilla/beTRUSTed_Root_CA_-_RSA_Implementation.crt
  error adding mozilla/thawte_Primary_Root_CA.crt
  error adding signet.pl/signet_ca1_pem.crt
  error adding signet.pl/signet_ca2_pem.crt
  error adding signet.pl/signet_ca3_pem.crt
  error adding signet.pl/signet_ocspklasa2_pem.crt
  error adding signet.pl/signet_ocspklasa3_pem.crt
  error adding signet.pl/signet_pca2_pem.crt
  error adding signet.pl/signet_pca3_pem.crt
  error adding signet.pl/signet_rootca_pem.crt
  error adding signet.pl/signet_tsa1_pem.crt
  error adding spi-inc.org/spi-ca-2003.crt
  error adding spi-inc.org/spi-cacert-2008.crt
  error adding telesec.de/deutsche-telekom-root-ca-2.crt
failed (VM used: java-6-openjdk).
dpkg: error processing ca-certificates-java (--configure):
 subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
 ca-certificates-java
E: Sub-process /usr/bin/dpkg returned an error code (1)

/proc/user_beancounters:

Version: 2.5
   uid  resource                     held              maxheld              barrier                limit              failcnt
12803:  kmemsize                  4612796              5723362           2147483646           2147483646                    0
        lockedpages                     0                    0               999999               999999                    0
        privvmpages                 76205                83773               262144               262144                    0
        shmpages                      640                  690               131072               131072                    0
        dummy                           0                    0                    0                    0                    0
        numproc                        36                   43               999999               999999                    0
        physpages                   22308                23091                    0           2147483647                    0
        vmguarpages                     0                    0               131072           2147483647                    0
        oomguarpages                22308                23091               131072           2147483647                    0
        numtcpsock                     15                   31              7999992              7999992                    0
        numflock                        5                    8               999999               999999                    0
        numpty                          1                    1               500000               500000                    0
        numsiginfo                      0                    6               999999               999999                    0
        tcpsndbuf                  262560              7030184            214748160            396774400                    0
        tcprcvbuf                  245760               507904            214748160            396774400                    0
        othersockbuf                20952                95288            214748160            396774400                    0
        dgramrcvbuf                     0                12848            214748160            396774400                    0
        numothersock                   16                   23              7999992              7999992                    0
        dcachesize                      0                    0           2147483646           2147483646                    0
        numfile                      1233                 1956             23999976             23999976                    0
        dummy                           0                    0                    0                    0                    0
        dummy                           0                    0                    0                    0                    0
        dummy                           0                    0                    0                    0                    0
        numiptent                      24                   24               999999               999999                    0

Update

I just gave this another go in light of the recent activity here. I don't totally remember what I was doing when I caused this error originally, but I did notice I was a few versions behind on Ubuntu itself. I ran an update on that and now I seem to have ca-certificates-java successfully installed. I'm not sure if there's something more nuanced going on, but I did confirm what Alex said, that ca-certificates-java and openjdk-6-jre seem depend on each other. What's curious is it seemed like the OS upgrade fixed the issue on its own. I was updating from the original install from my host: maybe they're flashing my VM with some install that's broken in this regard or something of the like.

The issue now seems to be more normal at least. Attempting to compile (or run from compiling elsewhere) even a simple hello world program fails with the following message:

user@domain:~# java HelloWorldApp
Picked up _JAVA_OPTIONS: -Xms128m -Xmx512m
Error occurred during initialization of VM
java.lang.OutOfMemoryError: unable to create new native thread
    at java.lang.Thread.start0(Native Method)
    at java.lang.Thread.start(Thread.java:614)
    at java.lang.ref.Reference.<clinit>(Reference.java:162)

This happens regardless of whether or not the _JAVA_OPTIONS environment variable has been set.


Solution 1:

Stephen,

Set _JAVA_OPTIONS prior to running the apt-get routines, by running the following at the command line: export _JAVA_OPTIONS="-Xms128m -Xmx512m"

You must include the underscore, and set -Xmx to about 80% of available RAM, with -Xms to something lower than -Xmx.

In the event you still encounter issues, especially hanging, ensure that your environment has access to at least two CPU cores; java does not run well with a single CPU core. From inside of your container, you can run the command: cat /proc/cpuinfo

Solution 2:

One way I was once able to let Java over-provision memory is to add the following to /etc/security/limits.conf (may be different for non-Debian distos) and try testing again in a new login shell:

* soft memlock unlimited
* hard memlock unlimited

Hovwever, it's unlikely that this setting will actually help you. You should probably get hosting with a higher privvmpages resource.