How to search LaTeX -documents in Spotlight?
I have reproduced the problem now also in El Capitan with new components (motherboard, 3x SSD) in my Macbook Air 2013-mid. I have had clean installation of OS X (El Capitan, Yosemite) while doing the test. The latest OS X where the .tex search works is 10.8.5 in Spotlight. I do not see any .tex documents in my Spotlight search at the moment. Tests
- No current answer solves the problem permanently. In Yosemite, the problem became back after a while with some answers. No stable solution exists and little understanding what is the cause of the problem.
The selection of Settings configurations are not relevant in Spotlight because we can use directly the command mdfind in terminal, which is equivalent to Spotlight.
However, we have tried Default, minimal and Full settings in Spotlight.
For the tests, we use default Settings in OS X.
System and Applications
- Systems OS X Yosemite, OS X El Capitan
- TexShop.app is installed. (MacTeX 2015, mactex-20150613.pkg).
- Tex App: TextMate2-beta8
- Spotlight/Finder/any search app does not work with TeX files and MacTeX 2015
- Spotlight works as expected with some different settings and also with MacTeX 2015.
The command mdfind (version June 10, 2004) is equivalent to Spotlight search so a simple test which returns nothing unsuccessfully in my system
mdfind sulfas | grep tex
I think the problem is that the system does not recognize the fileformat .tex`. The first thing is to get the system recognize it.
tjt263's suggestion to search only .tex in Finder/Spotlight
Output of Spotlight .tex or the terminal command gives tex -files correctly in my system.
However, combining it to other terms like, by .tex diabetes, gives no tex -files anymore.
Importers obtained by mdimport -L:
2015-09-22 15:31:42.352 mdimport[8163:707] Paths: id(502) (
"/Library/Spotlight/iBooksAuthor.mdimporter",
"/Library/Spotlight/iWork.mdimporter",
"/Library/Spotlight/Microsoft Office.mdimporter",
"/System/Library/Spotlight/Application.mdimporter",
"/System/Library/Spotlight/Archives.mdimporter",
"/System/Library/Spotlight/Audio.mdimporter",
"/System/Library/Spotlight/Automator.mdimporter",
"/System/Library/Spotlight/Bookmarks.mdimporter",
"/System/Library/Spotlight/Chat.mdimporter",
"/System/Library/Spotlight/CoreMedia.mdimporter",
"/System/Library/Spotlight/Font.mdimporter",
"/System/Library/Spotlight/iCal.mdimporter",
"/System/Library/Spotlight/Image.mdimporter",
"/System/Library/Spotlight/iPhoto.mdimporter",
"/System/Library/Spotlight/iPhoto8.mdimporter",
"/System/Library/Spotlight/Mail.mdimporter",
"/System/Library/Spotlight/MIDI.mdimporter",
"/System/Library/Spotlight/Notes.mdimporter",
"/System/Library/Spotlight/PDF.mdimporter",
"/System/Library/Spotlight/PS.mdimporter",
"/System/Library/Spotlight/QuartzComposer.mdimporter",
"/System/Library/Spotlight/RichText.mdimporter",
"/System/Library/Spotlight/SystemPrefs.mdimporter",
"/System/Library/Spotlight/vCard.mdimporter",
"/Applications/Microsoft Office 2011/Microsoft Outlook.app/Contents/Library/Spotlight/Microsoft Outlook.mdimporter",
"/Applications/Xcode.app/Contents/Library/Spotlight/uuid.mdimporter",
"/Applications/DjView.app/Contents/Library/Spotlight/DjVu.mdimporter"
)
bmikes's tests
File notes.tex not searchable in Finder. I change it to notes.txt which I can now search in Finder. This proposes me that not Search -program problem (Spotlight, md..., whatever you use, as I described many times in comments). So tex -problem in the system. Outputs of the same file but with two different endings. Output of txt -file:
15:42:23:Desktop:masi$ mdls 8.9.2015.txt
kMDItemContentCreationDate = 2015-09-22 13:38:31 +0000
kMDItemContentModificationDate = 2015-09-22 13:38:31 +0000
kMDItemContentType = "public.plain-text"
kMDItemContentTypeTree = (
"public.plain-text",
"public.text",
"public.data",
"public.item",
"public.content"
)
kMDItemDateAdded = 2015-09-22 13:39:22 +0000
kMDItemDisplayName = "8.9.2015.txt"
kMDItemFSContentChangeDate = 2015-09-22 13:38:31 +0000
kMDItemFSCreationDate = 2015-09-22 13:38:31 +0000
kMDItemFSCreatorCode = ""
kMDItemFSFinderFlags = 0
kMDItemFSHasCustomIcon = 0
kMDItemFSInvisible = 0
kMDItemFSIsExtensionHidden = 0
kMDItemFSIsStationery = 0
kMDItemFSLabel = 0
kMDItemFSName = "8.9.2015.txt"
kMDItemFSNodeCount = 62503
kMDItemFSOwnerGroupID = 20
kMDItemFSOwnerUserID = 502
kMDItemFSSize = 62503
kMDItemFSTypeCode = ""
kMDItemKind = "Plain Text Document"
kMDItemLogicalSize = 62503
kMDItemPhysicalSize = 65536
Output of tex -file:
15:42:27:Desktop:masi$ mdls 8.9.2015.tex
kMDItemContentCreationDate = 2015-09-22 13:38:31 +0000
kMDItemContentModificationDate = 2015-09-22 13:38:31 +0000
kMDItemContentType = "org.tug.tex"
kMDItemContentTypeTree = (
"org.tug.tex",
"public.text",
"public.data",
"public.item",
"public.content"
)
kMDItemDateAdded = 2015-09-22 13:42:37 +0000
kMDItemDisplayName = "8.9.2015.tex"
kMDItemFSContentChangeDate = 2015-09-22 13:38:31 +0000
kMDItemFSCreationDate = 2015-09-22 13:38:31 +0000
kMDItemFSCreatorCode = ""
kMDItemFSFinderFlags = 0
kMDItemFSHasCustomIcon = 0
kMDItemFSInvisible = 0
kMDItemFSIsExtensionHidden = 0
kMDItemFSIsStationery = 0
kMDItemFSLabel = 0
kMDItemFSName = "8.9.2015.tex"
kMDItemFSNodeCount = 62503
kMDItemFSOwnerGroupID = 20
kMDItemFSOwnerUserID = 502
kMDItemFSSize = 62503
kMDItemFSTypeCode = ""
kMDItemKind = "TeX document"
kMDItemLogicalSize = 62503
kMDItemPhysicalSize = 65536
Wanted commands:
mkdir /Users/masi/Desktop/masi
18:07:56:~:masii$ cp /tmp/8.9.2015.tex /Users/masi/Desktop/masi/
18:08:09:~:masi$ cd /Users/masi/Desktop/masi/
18:08:16:masi:masi$ echo osteoporosis > test_file.txt
18:08:24:masi:masi$ mdfind osteoporosis |grep /Users/masi/Desktop/masi/
/Users/masi/Desktop/masi/test_file.txt
18:08:47:masi:masi$ mdls -name kMDItemContentType test_file.txt
kdItemContentType = (public.plain-text)
18:09:49:masi:masi$ mv test_file.txt test_file.tex
18:10:14:masi:masi$ mdfind osteoporosis | grep /Users/masi/Desktop/masi/
18:10:20:masi:masi$ mdls -name kMDItemContentType test_file.tex
kMDItemContentType = "org.tug.tex"
Current Diagnostic Tools Using and Discussing in Chat
Well-hidden command from Is it possible to query the launch services database for applications that will open an arbitrary file or UTI type?
/System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/LaunchServices.framework/Versions/A/Support/lsregister -dump | grep -n7 org.tug.tex
We found that the MacTeX 2015's TeXShop does not have TeX.mdimporter. Installing this gives on the last line of mdimport -L:
"/Applications/TeX/TeXShop.app/Contents/Library/Spotlight/TeX.mdimporter" )
Unsuccessful try with Patrix
mkdir -p ~/Library/Spotlight; ln -s /Applications/TeX/TeXShop.app/Contents/Library/Spotlight/TeX.mdimporter ~/Library/Spotlight/
after which rm ~/Library/Spotlight/TeX.mdimporter and we reseted the index of Spotlight by moving HardDisc to the Prevent -panel and quickly remove it, so taking now some time to index the system.
klanomath's answer results which worked one day until going back to the bad state
I get unstably some .tex files in mdfind. After one day use from the init of this proposal, the condition went back to the bad state:
- osteoporosis search returns the same files as one day ago, but I noticed that those files, which have not been visible in search matches recently and enough long time when last time edited, are not visible in the search
- only recently edited files shown, and actually sometimes even 3.5 months old files shown
Running 18:25:06:~:masi$ /usr/bin/mdimport -d3 /Users/masi/Dropbox/Internal\ Diseases/24.9.2015.tex and getting
(Debug) Import: file:/Users/masi/Dropbox/Internal Diseases/24.9.2015.tex plugin:file://localhost/Library/Spotlight/TeX.mdimporter/ /Library/Spotlight/TeX.mdimporter
(Info) Import: edu.uo.texshop.tex
(Debug) Import: Attributes for file /Users/masi/Dropbox/Internal Diseases/24.9.2015.tex: {
":MD:kMDExtendedImportDictionary" = {
};
"_kMDItemFinderLabel" = 0;
"com_apple_metadata_modtime" = 464803819;
kMDItemContentCreationDate = "2015-09-23 19:57:11 +0000";
kMDItemContentModificationDate = "2015-09-24 16:10:19 +0000";
kMDItemContentType = "edu.uo.texshop.tex";
kMDItemContentTypeTree = (
"edu.uo.texshop.tex",
"public.text",
"public.data",
"public.item",
"public.content",
"public.plain-text",
"org.tug.tex"
);
kMDItemDateAdded = "2015-09-23 19:57:11 +0000";
kMDItemDisplayName = {
"" = "24.9.2015.tex";
};
kMDItemKind = {
"" = "TeX File";
en = "TeX File";
pt = "Documento TeX";
};
kMDItemLogicalSize = 18220;
kMDItemPhysicalSize = 20480;
kMDItemTextContent = "\...";
}
2015-09-24 18:25:50.335 mdimport[6875:707] Imported '/Users/masi/Dropbox/Internal Diseases/24.9.2015.tex' of type 'edu.uo.texshop.tex' with plugIn /Library/Spotlight/TeX.mdimporter.
2015-09-24 18:25:50.336 mdimport[6875:707] Attributes: {
":MD:kMDExtendedImportDictionary" = {
};
"_kMDItemFinderLabel" = 0;
"_kMDItemStaticInterestScore" = "0.5";
"com_apple_metadata_modtime" = 464803819;
kMDItemContentCreationDate = "2015-09-23 19:57:11 +0000";
kMDItemContentModificationDate = "2015-09-24 16:10:19 +0000";
kMDItemContentType = "edu.uo.texshop.tex";
kMDItemContentTypeTree = (
"edu.uo.texshop.tex",
"public.text",
"public.data",
"public.item",
"public.content",
"public.plain-text",
"org.tug.tex"
);
kMDItemDateAdded = "2015-09-23 19:57:11 +0000";
kMDItemDisplayName = {
"" = "24.9.2015.tex";
};
kMDItemKind = {
"" = "TeX File";
en = "TeX File";
pt = "Documento TeX";
};
kMDItemLogicalSize = 18220;
kMDItemPhysicalSize = 20480;
kMDItemTextContent = "\...";
}
Guess about the origin of the problem: /Volumes/Disc/.Spotlight-V100/...
I investigated the dynamics of the memory at different time points by fdupes -rd /Volumes/Disc2/, the following sample is during indexing:
[1] /Users/masi/Dropbox/det2/plan.tex
[2] /Volumes/Disc2/.Spotlight-V100/Store-V2/440B4DE1-129F-46AB-9D58-14F473B0282A/Cache/0000/0000/0003/213630.txt
[3] /Volumes/Disc2/.Spotlight-V100/Store-V2/440B4DE1-129F-46AB-9D58-14F473B0282A/Cache/0000/0000/0003/213631.txt
[4] /Volumes/Disc2/.Spotlight-V100/Store-V2/440B4DE1-129F-46AB-9D58-14F473B0282A/Cache/0000/0000/0003/213633.txt
... [10] sometimes much more
---
where the file content is two empty lines. You see many duplicate entries created by Spotlight for its index. Spotlight's index has much duplicates in many points of its functioning. I also studied this behavior across different medias. There are a lot of dummy/duplicate files created during the indexing and during the stable period when the indexing is done. I think Apple has done bad work in designing the index which Spotlight is using.
How can you search LaTeX documents with mdfind/Spotlight/Finder?
Solution 1:
This likely is a result of the editor/software for TEX you use since you don't need any spotlight importer to process plain text files. (So you could just delete your app's custom importer or choose an update/different importer).
Rather than guess at the case - here's how to nail down where the problem lies. The editor could be changing the ** kMDItemContentType** to one that's not indexed or you have a third party spotlight extension that's crashing. Here's how I'd know which of these (or something more unexpected) is happening:
A) Does spotlight index an arbitrary text file when you change the extension from .txt to .tex?
B) Compare metadata for the files to know what's happening using mdls
To test A, open Text Edit and paste one word into the document: osteoporosis
If needed, convert the document to plain text (it's probably rich text) - Shift + Command + T (or use the format menu - Make Plain Text) and save it to your desktop as file.txt - If the format menu says Make Rich Text then don't press the keys and just save the document.
At that point, spotlight should see the file immediately. If not, you have spotlight problem and not .tex file problems. This is a very basic problem if your spotlight is broken so as not to index plain text files. If this work, then change the .txt extension to .tex and re-check spotlight.
For test B - use the mdls command to examine the metadata differences between your file and the TextEdit file that work with spotlight. Pay special attention to the following fields:
kMDItemContentType = "public.plain-text"
kMDItemContentTypeTree = (
"public.plain-text",
"public.text",
"public.data",
"public.item",
"public.content"
)
kMDItemKind = "Plain Text Document"
Changing the .txt to .tex causes a change to this (since I have no application that claims the file extension of .tex and maps it to a launch services/spotlight classification and proper kMDItemContentType/kMDItemKind as opposed to a generic and ad/hoc type:
kMDItemContentType = "dyn.ah62d4rv4ge81k3p2"
kMDItemContentTypeTree = (
"dyn.ah62d4rv4ge81k3p2",
"public.data",
"public.item"
)
kMDItemKind = "Document"
Here's a short test you can cut/paste if your terminal skills are not yet developed. It would delete the contents of any file names test_file on your desktop so make a backup if you're not sure before pasting the commands below:
cd ~/Desktop/
echo osteoporosis > test_file.txt
mdfind osteoporosis | grep Desktop
mdls -name kMDItemContentType test_file.txt
mv test_file.txt test_file.tex
mdfind osteoporosis | grep Desktop
mdls -name kMDItemContentType test_file.tex
The mdfind commands are the equivalent of using Spotlight, so check that the terminal search matches the spotlight search at that moment.
As a footnote, these tools are only needed to diagnose the extent of breakage and not a substitute for spotlight search in the end. Just that you can't diagnose a spotlight issue with spotlight alone. Also, you might want to watch the console log while you are doing the steps in ~/Desktop in case there are crash reports or other spotlight diagnostics happening while you are poking at the subsystem.
Also, things get nasty quickly if you can search the plain text document but not the text one. In your update, you mentioned that .tex files are of type "org.tug.tex" - you could explore the launch services database to find out what importers the system things is responsible for parsing this file and remove them (or just make an educated guess and temporarily delete the TEX apps to see if it "fixes" spotlight temporarily).
lsregister -dump| grep -n7 org.tug.tex
Where lsregister is a well-hidden tool in /System - see this thread on SuperUser to read up on launch services: https://superuser.com/questions/323599/ and this thread here: Duplicate entries in "Open With" menu in Finder even after rebuilding Launch Services for some oddball things to try. I'm fairly confident I could suss out what's amiss on your system in about 20 minutes of poking, but writing down all the possibilities in a way accessible to you and guessing a bit of the answers is proving harder than I expected.
Solution 2:
I tried to reproduce your Spotlight problems in a virtual machine set-up with Mac OS X 10.8.
After installing MacTex-20150613 and updating all components (>300) including TexShop and LaTeXiT I had no problems searching for content in *.tex files.
After entering:
/System/Library/Frameworks/CoreServices.framework/Frameworks/LaunchServices.framework/Support/lsregister -kill -r -domain local -domain system -domain user
I "succeeded" losing the ability to search for content in *.tex files.
To regain the ability to do so I did the following:
-
Logged-in as admin I copied the file /Applications/TeX/TeXShop.app/Contents/Library/Spotlight/TeX.mdimporter to /Library/Spotlight/ with
sudo cp -R /Applications/TeX/TeXShop.app/Contents/Library/Spotlight/TeX.mdimporter /Library/Spotlight/ - Prepared the reindexing of my system by first going to the folder /.Spotlight-V100 (in the Finder menubar -> Go -> Go to Folder... and enter /.Spotlight-V100) and moving all files and folders in this folder to the trash can. You have to enter your admin password to do so. If you fail to enter the folder log in as root (You may have to enable root first using /System/Library/CoreServices/Directory Utility.app) and remove the content of the folder as root.
- Rebooted the Mac
- Logged in as admin and emptied the trash can. The reindexing will be started automatically.
Continuously checking all relevant informations like file attributes or launch services in different states of the system (with several VM snapshots) using mdls, xattr and other tools didn't reveal anything interesting or suspicious.
I have repeated all steps above more thoroughly in a new VM and used mdimport -d3 to check the behavior of Spotlight.
In a vanilla Mac OS X a .tex file is properly imported using a dyn.random importer:
host:~user$ /usr/bin/mdimport -d3 /Users/user/Desktop/sample.tex
(Debug) Import: file:/Users/user/Desktop/sample.tex plugin:(null) (null)
2015-09-23 20:20:42.359 mdimport[313:707] Imported '/Users/user/Desktop/sample.tex' of type 'dyn.ah62d4rv4ge81k3p2' with no plugIn.
The content (keywords) was found.
The same applies to Mac OS X after installing but not starting one of the Tex.apps, the importer changed to RichText.mdimporter though:
host:~ user$ /usr/bin/mdimport -d3 /Users/user/Desktop/sample.tex
(Debug) Import: file:/Users/user/Desktop/sample.tex plugin:file://localhost/System/Library/Spotlight/RichText.mdimporter/ /System/Library/Spotlight/RichText.mdimporter
(Info) Import: public.tex
After starting a Tex.app (in my case TeXShop) the importer uses the default TeX.mdimporter residing in the depth of the TeXShop bundle:
host:~ user$ /usr/bin/mdimport -d3 /Users/user/Desktop/sample.tex
(Debug) Import: file:/Users/user/Desktop/sample.tex plugin:file://localhost/Applications/TeX/TeXShop.app/Contents/Library/Spotlight/TeX.mdimporter/ /Applications/TeX/TeXShop.app/Contents/Library/Spotlight/TeX.mdimporter
(Info) Import: edu.uo.texshop.tex
After updating the MacTex-20150613 including TexShop and LaTeXiT the indexing is completely broken:
host:~ user$ /usr/bin/mdimport -d3 /Users/user/Desktop/sample-2.tex
(Debug) Import: file:/Users/user/Desktop/sample-2.tex plugin:(null) (null)
2015-09-23 21:29:49.278 mdimport[5541:707] Imported '/Users/user/Desktop/sample-2.tex' of type 'public.tex' with no plugIn.
The proper path to the plugin TeX.mdimporter (inside TexShop) got lost and so the capability to index a new *.tex file vanished. Other plugins like the dyn.random or RichText.mdimporter don't step in probably because they don't index files of type 'public.tex'. Content search in (at least new) .tex files isn't possible anymore.
After copying TeX.mdimporter to /Library/Spotlight content indexing is restored:
host:~ user$ /usr/bin/mdimport -d3 /Users/user/Desktop/sample-3.tex
(Debug) Import: file:/Users/user/Desktop/sample-3.tex plugin:file://localhost/Library/Spotlight/TeX.mdimporter/ /Library/Spotlight/TeX.mdimporter
(Info) Import: public.tex
To reindex all .tex files properly (especially those where content keywords aren't searchable) you have to rebuild the spotlight database (check the first part of my answer) or change the modification dates of all .tex files to initiate the reindexing of them.
Solution 3:
Here's how I solved this problem.
Note. This still works since I first posted the solution. At that time I was using Mac OS 10.11.5 and TeXShop 3.62; now it's Mac OS 10.12.3 and TeXShop 3.77. To edit the .plist files (as explained below) I use Xcode.
First, using the mdls command, I found that all of my .tex files reported kMDItemContentType = “public.tex” (If your system reports differently, then you would have to modify the following instructions accordingly.)
So I opened TeXShop>Contents (by right-clicking TeXShop and selecting Show Package Contents). I amended Info.plist by adding public.tex as a Document Content Type UTI as follows (see the line highlighted in blue):
Then I went on to amend the Info.plist in TeXShop>Contents>Library>Spotlight>TeX.mdimporter>Contents (execute Show Package Contents of TeX.mdimporter). Just like above, I added public.tex in appropriate places. See the next picture:
After this, Spotlight is searching the contents of .tex files.
Solution 4:
This seems to be a problem with Spotlight MetaData import. A missing, corrupted or broken path to the "Spotlight Plugin" installed by the "TexLive" package.
The output of "mdimport -L" on my machine includes the line:
"/Applications/TeX/TeX Live Utility.app/Contents/Library/Spotlight/DVIImporter.mdimporter"
I don't see a similar line in your output. A good solution for you might be to reinstall "TeX Live" distribution, from The MacTeX-2015 Distribution at: https://www.tug.org/mactex/ this should correct your broken mdimport path.
mactex-20150613.pkg
http://www.macissues.com/2015/04/03/faulty-spotlight-plugins-may-cause-incomplete-searches/
http://www.macissues.com/2014/12/12/how-to-determine-when-your-spotlight-index-needs-to-be-rebuilt/
Solution 5:
If you can be without Spotlight
If you want to get a list of files and then browse each of them separately.
- Use script
haetexas described in the thread How to search .tex files? which works also in BSD because POSIX.
If you want to get all filenames and matches in the same view, use for searching all files in Directory Time... by traversing recursively
-
files/symlinked files [fd0]
find -L . -type d -path './Time*' \ -exec grep -d "recurse" series {} + | cut -c -80 -
.tex files without spaces in directoryNames (source)
find -L $(find . -type l -name 'Math*') -name '*.tex' \ -exec fgrep word /dev/null {} + | cut -c -80
If Spotlight
Temporary fix is to use .txt documents instead of .tex documents in some folders because it is a stable solution
cat *.tex > summary_all.txt
Other attempts
- I am trying to find a terminal approach for searching .tex files here about To Open Find's resulted PATH by Click in Viewer?
- I am trying to replace
mdfindhere about How to Replace mdfind with any other tool for .tex?