When should objects be used in OCaml?

As a general rule of thumb, don't use objects. The additional complexity they bring in is not that often worth it. I think that's a rule that apply to other languages as well, but that's another story. At least with OCaml one can objectively (no pun intended) say that the common practice is to not use object except in rare cases.

Object provide a bundle of:

1. A "standard" style for carrying around and using records of functions, with possibly polymorphic types
2. Facilities for open recursion through self (implementation inheritance)
3. A structural, extensible product type with row polymorphism (for open object types) and subtyping (for closed object types)

You may use any of those together, or separately.

In my experience, point (1) alone is not particularly worth using objects: you can just use records of functions and it is just as clear.

Uses Cases of Open Recursive / Inheritance

Point (2) on the contrary is a good justification for using an object-oriented style; it is used in this manner by Camlp4 for example: Camlp4 defines classes that fold over an AST doing nothing, and you can inherit this traversal object to implement the behavior you want on only the syntactic constructions you want (and defer the boring traversal plumbing to your mother class).

For example, one may extend the Camlp4Ast.map object, that defines a simple map function on the Camlp4 representation of the OCaml Abstract Syntax Trees, just mapping every construct to itself, recursively. If you want to, say, map all (fun x -> e1) e2 expressions to let x = e2 in e1, you inherit from this object, and override the expr method, handling only the case you want (left-hand-side is a function), delegating the other to the inherited behavior. This will give you an object that knows how to apply this transform over a complete program, recursively, without you having to write any boilerplate code; and you can further extend this transformation with additional behavior if you so wish.

Fun with object types

Point (3) is also a justification for using objects as "extensible records", or as "type-level arrays"; some libraries using object types, but not object at runtime: they use object types as phantom types to carry around information, benefiting from the richer type-level operations you can have on objects. Besides, structural typing allow for different authors to have compatible types without strong dependencies to a common component defining the (nominal) types they share; objects have been used for standardization of input/output components for example.

A not-uncommon, very simple use case of this is an idiomatic way to represent types that have a lot of parameters. Instead of writing:

type ('name, 'addr, 'job, 'id) person = ....
val me : (string, string, Job.t, Big_int.big_int) person

you can use object types as structural "type-level records" to write instead:

type 'a person = .... constraint 'a = < name:'n; addr:'a; job:'j; id:'i >
val me : < name:string; addr:string; job:Job.t; id:Big_int.big_int > person

For the more advanced use of object types as phantom types, you can have a look at the ShCaml (doc) library (where it is used to represent which shell commands a string input is compatible with) by Alec Heller and Jesse Tov, or my own Macaque library (doc and api doc), which uses object types to represent SQL values (including nullability information) and table row types.
Polymorphic variants (another advanced feature of OCaml type system; in a sentence, the relation between objects and records is the same as the relation between polymorphic variants and algebraic sum types).have also been used as phantom types, for example in this simple example by Richard Jones, or to check validity of HTML documents in the Ocsigen framework.

Beware however that those advanced type hackeries come at a significant complexity cost; before using them, you must carefully balance it with the additional expressivity and static safety that they bring.

Summing up

• as a base assumption, you're safe with not using objects at all; you should only introduce them in your design if you feel you're missing something, not by default

• objects are convenient for open recursion / inheritance: refining a behavior that is already defined in the default/boring cases

• structural typing is occasionally useful when you want to reason on values independently providing a set of features/capacities