Preventing System.exit() from API

I am using a third party library that does a System.exit() if it encounters exceptions. I am using the APIs from a jar. Is there anyway that I can prevent the System.exit() call because it causes my application to shutdown? I cannot decompile and recompile the jar after removing the System.exit() because of a lot of other licensing issues. I once came across an answer [to some other question that I do not remember] in stackoverflow that we can use the SecurityManager in Java to do something like this.

Solution 1:

There is a blog post here,

http://jroller.com/ethdsy/entry/disabling_system_exit

Basically it installs a security manager which disables System.exit() with code from here,

  private static class ExitTrappedException extends SecurityException { }

  private static void forbidSystemExitCall() {
    final SecurityManager securityManager = new SecurityManager() {
      public void checkPermission( Permission permission ) {
        if( "exitVM".equals( permission.getName() ) ) {
          throw new ExitTrappedException() ;
        }
      }
    } ;
    System.setSecurityManager( securityManager ) ;
  }

  private static void enableSystemExitCall() {
    System.setSecurityManager( null ) ;
  }

Edit: Max points out in comments below that

as of Java 6, the permission name is actually "exitVM."+status, e.g. "exitVM.0".

However, the permission exitVM.* refers to all exit statuses, and exitVM is retained as a shorthand for exitVM.*, so the above code still works (see the documentation for RuntimePermission).

Solution 2:

See my reply to How to avoid JFrame EXIT_ON_CLOSE operation to exit the entire application?.

Edit 1: The source that was linked. Demonstrates how to use a SecurityManager to prevent System.exit(n).

import java.awt.*;
import java.awt.event.*;
import java.security.Permission;

/** NoExit demonstrates how to prevent 'child'
applications from ending the VM with a call
to System.exit(0).
@author Andrew Thompson */
public class NoExit extends Frame implements ActionListener {

  Button frameLaunch = new Button("Frame"),
     exitLaunch = new Button("Exit");

  /** Stores a reference to the original security manager. */
  ExitManager sm;

  public NoExit() {
     super("Launcher Application");

     sm = new ExitManager( System.getSecurityManager() );
     System.setSecurityManager(sm);

     setLayout(new GridLayout(0,1));

     frameLaunch.addActionListener(this);
     exitLaunch.addActionListener(this);

     add( frameLaunch );
     add( exitLaunch );

     pack();
     setSize( getPreferredSize() );
  }

  public void actionPerformed(ActionEvent ae) {
     if ( ae.getSource()==frameLaunch ) {
        TargetFrame tf = new TargetFrame();
     } else {
        // change back to the standard SM that allows exit.
        System.setSecurityManager(
           sm.getOriginalSecurityManager() );
        // exit the VM when *we* want
        System.exit(0);
     }
  }

  public static void main(String[] args) {
     NoExit ne = new NoExit();
     ne.setVisible(true);
  }
}

/** This example frame attempts to System.exit(0)
on closing, we must prevent it from doing so. */
class TargetFrame extends Frame {
  static int x=0, y=0;

  TargetFrame() {
     super("Close Me!");
     add(new Label("Hi!"));

     addWindowListener( new WindowAdapter() {
        public void windowClosing(WindowEvent we) {
           System.out.println("Bye!");
           System.exit(0);
        }
     });

     pack();
     setSize( getPreferredSize() );
     setLocation(++x*10,++y*10);
     setVisible(true);
  }
}

/** Our custom ExitManager does not allow the VM
to exit, but does allow itself to be replaced by
the original security manager.
@author Andrew Thompson */
class ExitManager extends SecurityManager {

  SecurityManager original;

  ExitManager(SecurityManager original) {
     this.original = original;
  }

  /** Deny permission to exit the VM. */
   public void checkExit(int status) {
     throw( new SecurityException() );
  }

  /** Allow this security manager to be replaced,
  if fact, allow pretty much everything. */
  public void checkPermission(Permission perm) {
  }

  public SecurityManager getOriginalSecurityManager() {
     return original;
  }
}

Solution 3:

The previous code sample is partially correct, but I found that it ended up blocking my code's access to files. To get around that problem, I wrote my SecurityManager a little differently:

public class MySecurityManager extends SecurityManager {

private SecurityManager baseSecurityManager;

public MySecurityManager(SecurityManager baseSecurityManager) {
    this.baseSecurityManager = baseSecurityManager;
}

@Override
public void checkPermission(Permission permission) {
    if (permission.getName().startsWith("exitVM")) {
        throw new SecurityException("System exit not allowed");
    }
    if (baseSecurityManager != null) {
        baseSecurityManager.checkPermission(permission);
    } else {
        return;
    }
}

}

In my case, I needed to prevent a 3rd party library from terminating the VM. But there were also some grails tests that were calling System.exit. So, I wrote my code so that it only activated the custom security manager immediately before the call to the 3rd party library (not a common event) and then immediately restored the original security manager, if any, afterwards.

It's all a little ugly. Ideally, I would have preferred to simply remove the System.exit code, but I do not have access to the 3rd party library's source code.