Use a node module from casperjs

Is it possible to install a node module, installed via npm, and then require it from a casperjs script?

(I see lots of posts and tools for running casper or phantom from inside node.js, but that is not what I'm trying to do.)

The casperjs docs seem to say it is possible, but only show with hand-written toy modules that don't really do anything. The real-world module I'm trying to install is imap, but at this point I cannot get any module to work, even built-in ones like net. Simple example:

npm install imap
echo "var test = require('imap');" > test.js
casperjs test.js

Gives me:

CasperError: Can't find module imap

/usr/local/src/casperjs/bin/bootstrap.js:263 in patchedRequire
test.js:1

(I can see the imap module from npm ls, and I can use it fine from a node.js script.)

Or alternatively with a built-in module:

echo "var test = require('net');" > test.js
casperjs test.js

Complains "Can't find module net"

I have npm --version of 1.4.14 and nodejs --version of v0.10.29. Are either of those too old, I wonder? (Casper is 1.1.0-beta, and Phantom is 1.9.7, both of which are the latest versions.)

Solution 1:

PhantomJS and SlimerJS (the engines that are used for CasperJS) are not Node.js modules. They can be installed through npm for convenience. They have a different base infrastructure of modules which is distinct from Node.js.

You will not be able to use imap or any module that depends on the net module. As Fanch points out, there are modules that can work inside the phantomjs runtime.

If a module only uses some functionality of some native node.js module, you could try to change the implementation to use the API that phantomjs provides. I don't think this is easy though. Most of the time you will run into a wall.

In the case of imap, it is pretty hopeless. You cannot even re-implement the require("net").Socket, because WebSockets are not supported in phantomjs (at least in 1.9.7).

Solution 2:

Here an example with the colors module :

var colors = require('colors');


 casper.test.begin('\n*Colors module*\n', function suite(test) {
    casper.start()
    .thenOpen('https://www.google.fr/', function() {
        console.log("test require\n".zebra);
        console.log("test require\n".rainbow);
        console.log("test require\n".red.underline.bold);
      })
    .run(function() {
            test.done();
    });
});

• node-modules
  • colors
• testnode.js

casperjs test testnode.js

output :

It seems it's not as simple when the required module has dependencies.

Solution 3:

In my case, I wanted to load underscorejs. Underscore is a series of functions and don't have complicated interactions with javascript objects, so there is no problem just requiring the javascript file and then having access to its functions.

I started by finding the root to my nodejs installation (from the CLI):

node --help

Which led me to finding my node path:

echo $NODE_PATH

Which was at:

/usr/local/lib/node_modules/

Underscore was at

/usr/local/lib/node_modules/underscore/underscore.js

So my final require statement in my CasperJS script was.

var _ = require('/usr/local/lib/node_modules/underscore/underscore.js');

Now in my script I test to see if underscorejs is loaded:

this.echo(_.now());

And I see the current time.

CAVEAT: Since this is running asynchronously, if you put your _.now() statement right after the require, it will give you an undefined object error. As a note, I'm using Casper 1.1, which uses PhantomJS's native require() function. If you are using < Casper 1.1, I don't think require will work.

UPDATE: Since this is the case, I use CasperJS then() function to load my utilities synchronously, making sure to declare my variables within the global scope. Here's how that looks:

//at the top of the file-ish, declare variables that will hold loaded libraries.
var utils, _;
var casper = require('casper').create(); //create casper
casper.start('http://example.com'); //start casper at URL.

casper.then(function loadRequires(){ //load the requirements
    utils = require('utils', function(){this.echo('utils loded')});
    _ = require('/usr/local/lib/node_modules/underscore/underscore.js');
});

casper.then(function myAwesomeStuff() {
    this.echo(_.now()); //now, access the loaded requirements
    utils.dump('this stuff is soooo awesome!!!!!!!!!!!!!!!!!!!!');
    //do stuff on the page you opened in the start function here.
    );
});

You can read more about the Casper prototype and the then() method at the API docs: http://casperjs.readthedocs.org/en/latest/modules/casper.html#casper-prototype