How do I collect all of my notes and highlights from iBooks?
I've got a lot of highlighting and notes in iBooks that I've read, and would like to be able to collect them in one easy to use and manipulate format (e.g for writing papers and citing quotes).
For example, I'd like a highlight like this
to produce something (e.g., in CSV) like
Quod me nutrit me destruit – that which sustains me also destroys me,14,Tamburlane Parts One and Two,Christopher Marlowe,Anthony B. Dawson ed.,Bloomsbury,https://itun.es/us/qSrZ0.l
I can see how to do this (sort of) laboriously, one note at a time, using iBook's "sharing" feature (or copy and paste, of course) but I don't see any way to do it in bulk, for all of my notes from a book, or even all my books.
Is there a way to accomplish this, with an Apple Script or using Automator for example? Or perhaps there is a text or XML file containing my notes and highlighting that I could write a script (in Python, preferably) to parse.
Solution 1:
iBooks doesn't have AppleScript support. The annotations are stored in a SQLite file: ~/Library/Containers/com.apple.iBooksX/Data/Documents/AEAnnotation/.
You could try to parse that. This answer gives a link to Digested, which reads that database and then allows you to export your annotations to Evernote, but I don't know what formatting they'll have or if you want to mess with Evernote.
One (possibly) simple solution would be to open the book in iBooks for iOS. You could then email the annotations in bulk to yourself.
- Open the book
- Press the "list button" to bring up the Table of Contents
- Switch to the Notes tab
- Press the Share button
- Select Edit Notes
- Select All
- Share via email.
Edit:
Actually, after reading a comment on reddit, there seems to be a way to export them all from iBooks on OS X as well:
You can export your notes in an email from Notes -> Select All -> Share (you need to hold ctrl while you right click to retain your selection). Your highlighted portions will be copied into the email with your notes and formatted nicely. Strangely, on the Mac the application doesn't care about whether the book is copy protected—it will always copy the highlighted portion. The iOS application indeed discriminates though. If your book is copy protected, then only the chapter name will be shared. That seems to be the only way to do it unfortunately. :/
Using my laptop's trackpad, I had to hold down ctrl + shift while tapping on the trackpad in order to bring up the contextual menu while retaining the selection.
Solution 2:
I've written a script for this purpose that extracts the notes from your Mac and outputs Evernote export files, ready for double clicking. Perhaps you could modify my script if it doesn't suit your purposes precisely.
In short, it reads the SQLite databases in: ./Library/Containers/com.apple.iBooksX/Data/Documents/BKLibrary ./Library/Containers/com.apple.iBooksX/Data/Documents/AEAnnotations
... and in this case, exports them to Evernote's .enex format.
https://github.com/jorisw/ibooks2evernote/
<?php
/*
* iBooks notes to Evernote converter
* by Joris Witteman <[email protected]>
*
* Reads the iBooks Annotations library on your Mac and exports
* them, tagged with their respective book title and imported in
* separate notebooks.
*
* Usage:
*
* Move this script to the top of your personal home directory on your Mac.
* This is the folder that has your name, which the Finder opens if you
* click on the Finder icon in the Dock.
*
* To export your notes to Evernote:
*
* 1. Run the following command in the Terminal:
*
* php ./ibooks2evernote.php
*
* 2. Open the newly created "iBooks exports for Evernote" folder from your
* home folder, open each file in there, Evernote will open and start
* importing your notes.
*
*/
// Default file locations for required iBooks data
define('RESULT_DIRECTORY_NAME',"iBooks exports for Evernote");
define('BOOKS_DATABASE_DIRECTORY','./Library/Containers/com.apple.iBooksX/Data/Documents/BKLibrary');
define('NOTES_DATABASE_DIRECTORY','./Library/Containers/com.apple.iBooksX/Data/Documents/AEAnnotation');
if(file_exists(RESULT_DIRECTORY_NAME)){
die("The destination folder for the exports already exists on your Mac.\nPlease move that one out of the way before proceeding.\n");
}
// Verify presence of iBooks database
if(!file_exists(BOOKS_DATABASE_DIRECTORY)){
die("Sorry, couldn't find an iBooks Library on your Mac. Have you put any books in there?\n");
}else{
if(!$path = exec('ls '.BOOKS_DATABASE_DIRECTORY."/*.sqlite")){
die("Could not find the iBooks library database. Have you put any books in there?\n");
}else{
define('BOOKS_DATABASE_FILE',$path);
}
}
// Verify presence of iBooks notes database
if(!file_exists(NOTES_DATABASE_DIRECTORY)){
die("Sorry, couldn't find any iBooks notes on your Mac. Have you actually taken any notes in iBooks?\n");
}else{
if(!$path = exec('ls '.NOTES_DATABASE_DIRECTORY."/*.sqlite")){
die("Could not find the iBooks notes database. Have you actually taken any notes in iBooks?\n");
}else{
define('NOTES_DATABASE_FILE',$path);
}
}
// Fire up a SQLite parser
class MyDB extends SQLite3
{
function __construct($FileName)
{
$this->open($FileName);
}
}
// Retrieve any books.
$books = array();
$booksdb = new MyDB(BOOKS_DATABASE_FILE);
if(!$booksdb){
echo $booksdb->lastErrorMsg();
}
$res = $booksdb->query("
SELECT
ZASSETID,
ZTITLE AS Title,
ZAUTHOR AS Author
FROM ZBKLIBRARYASSET
WHERE ZTITLE IS NOT NULL");
while($row = $res->fetchArray(SQLITE3_ASSOC) ){
$books[$row['ZASSETID']] = $row;
}
$booksdb->close();
if(count($books)==0) die("No books found in your library. Have you added any to iBooks?\n");
// Retrieve the notes.
$notesdb = new MyDB(NOTES_DATABASE_FILE);
if(!$notesdb){
echo $notesdb->lastErrorMsg();
}
$notes = array();
$res = $notesdb->query("
SELECT
ZANNOTATIONREPRESENTATIVETEXT as BroaderText,
ZANNOTATIONSELECTEDTEXT as SelectedText,
ZANNOTATIONNOTE as Note,
ZFUTUREPROOFING5 as Chapter,
ZANNOTATIONCREATIONDATE as Created,
ZANNOTATIONMODIFICATIONDATE as Modified,
ZANNOTATIONASSETID
FROM ZAEANNOTATION
WHERE ZANNOTATIONSELECTEDTEXT IS NOT NULL
ORDER BY ZANNOTATIONASSETID ASC,Created ASC");
while($row = $res->fetchArray(SQLITE3_ASSOC) ){
$notes[$row['ZANNOTATIONASSETID']][] = $row;
}
$notesdb->close();
if(count($notes)==0) die("No notes found in your library. Have you added any to iBooks?\n\nIf you did on other devices than this Mac, make sure to enable iBooks notes/bookmarks syncing on all devices.");
// Create a new directory and cd into it
mkdir(RESULT_DIRECTORY_NAME);
chdir(RESULT_DIRECTORY_NAME);
$i=0;
$j=0;
$b=0;
foreach($notes as $AssetID => $booknotes){
$Body = '<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE en-export SYSTEM "http://xml.evernote.com/pub/evernote-export3.dtd">
<en-export export-date="'.@strftime('%Y%m%dT%H%M%S',time()).'" application="iBooks2Evernote" version="iBooks2Evernote Mac 0.0.1">';
$BookTitle = $books[$AssetID]['Title'];
$j = 0;
foreach($booknotes as $note){
$CappedText = null;
$TextWithContext = null;
// Skip empty notes
if(strlen($note['BroaderText']?$note['BroaderText']:$note['SelectedText'])==0) continue;
$HighlightedText = $note['SelectedText'];
// Cap the titles to 255 characters or Evernote will blank them.
if(strlen($HighlightedText)>255) $CappedText = substr($note['SelectedText'],0,254)."…";
// If iBooks stored the surrounding paragraph of a highlighted text, show it and make the highlighted text show as highlighted.
if(!empty($note['BroaderText']) && $note['BroaderText'] != $note['SelectedText']){
$TextWithContext = str_replace($note['SelectedText'],"<span style=\"background: yellow;\">".$note['SelectedText']."</span>",$note['BroaderText']);
}
// Keep some counters for commandline feedback
if($j==0)$b++;
$i++;
$j++;
// Put it in Evernote's ENEX format.
$Body .='
<note><title>'.($CappedText?$CappedText:$HighlightedText).'</title><content><![CDATA[<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE en-note SYSTEM "http://xml.evernote.com/pub/enml2.dtd">
<en-note>
<div>
<p>'.($TextWithContext?$TextWithContext:$HighlightedText).'</p>
<p><span style="color: rgb(169, 169, 169);font-size: 12px;">From chapter: '.$note['Chapter'].'</span></p>
</div>
<div>'.$note['Note'].'</div>
</en-note>
]]></content><created>'.@strftime('%Y%m%dT%H%M%S',@strtotime("2001-01-01 +". ((int)$note['Created'])." seconds")).'</created><updated>'.@strftime('%Y%m%dT%H%M%S',@strtotime("2001-01-01 +". ((int)$note['Modified'])." seconds")).'</updated><tag>'.$BookTitle.'.</tag><note-attributes><author>[email protected]</author><source>desktop.mac</source><reminder-order>0</reminder-order></note-attributes></note>';
}
$Body .='
</en-export>
';
file_put_contents($BookTitle.".enex", $Body);
}
echo "Done! Exported $i notes into $b separate export files in the '".RESULT_DIRECTORY_NAME."' folder.\n\n";
Solution 3:
- Install the free DB Browser for SQLite.
- Go to the iBooks annotations folder:
~/Library/Containers/com.apple.iBooksX/Data/Documents/AEAnnotation/ - Copy the
.sqlitefile somewhere (like Desktop) to keep the original safe. - Open the file with DB Browser.
- Find some notes in your target book by browsing the data.
- Filter by the the ZANNOTATIONASSETID to show only notes in the target book.
- Copy and paste the annotations you want into Numbers or whatever application you prefer.