Managing Index Size
Exclude unnecessary files and folders from repo to index faster!
What is Indexable Size?
Indexable size is size of all code files, excluding following from the folder:
Directory/File based filtering
logs, node_modules, dist, target, bin, package-lock.json, data.json, build, .gradle, .idea, gradle, extension.js, vendor.js, ngsw.json, polyfills.js, ngsw-worker.js, runtime.js, runtime-main.js, service-worker.js, bundle.js, bundle.css
Extension based filtering
bin, exe, dll, log, aac, avif, bmp, cda, gif ,mp3, mp4, mpeg, weba, webm, webp, oga, ogv, png, jpeg, jpg, bmp, wpa, tif, tiff, svg, ico, wav, mov, avi, doc, docx, ppt, pptx, xls, xlsx, ods, odp, odt, pdf, epub, rar, tar, zip, vsix, 7z, bz, bz2, gzip, jar, war, gz, tgz, woff, woff2, eot, ttf, map, apk, app, ipa, lock, tmp, logs, gmo, pt
Hidden files are filtered i.e., files starting with "."
All Empty files are filtered.
All Binary files are also filtered.
Is there any limit on repository size?
For workspaces that have upgraded to Bito's 10X Developer Plan, we have set the indexable size limit to 120MB per repo. However, once we launch the "AI that Understands Your Code" feature for our Forever Free users, they will be restricted to repositories with an indexable size limit of 10MB.
Learn more about indexable size above and see which files and folders are excluded by default.
You can reduce your repo's indexable size by excluding certain files and folders from indexing using .bitoignore file and remain within the limit.
What if a repo hits 120MB limit?
If a repo hits 120MB limit, then the below error message will be displayed in the "Manage repos" tab and the repo's index status will be changed to "Not Indexed".
Sorry, we don’t currently support repos of this size. Please use .bitoignore to reduce the size of the repo you want Bito to index.
To fix this issue, follow our instructions regarding how to use .bitoignore file and reduce your repo's size and bring it under the max limit of 120MB.
After that, you must delete the index and then restart the indexing by clicking on the "Start Indexing" button shown for the repo folder. You can also follow our step-by-step guides to Start Indexing in Visual Studio Code and Start Indexing in JetBrains IDEs.
What is .bitoignore and how to use it?
A .bitoignore
file is a plain text file where each line contains a pattern or rules that tells Bito which files or directories to ignore and not index. In other words, it's a way to reduce your repo's indexable size. You can also see which files/folders are excluded by default.
There are two ways to use .bitoignore
file:
Create a
.bitoignore
file inside the folder where indexes created by Bito are stored. e.g.<user-home-directory>/.bito/localcodesearch/.bitoignore
On Windows, this path will be something like:
C:\Users\<your username>\.bito\localcodesearch\.bitoignore
Note: The custom ignore rules you set in this
.bitoignore
file will be applied to all the repositories where you have enabled indexing.
Create a
.bitoignore
file inside your repository's root folder.
If a .gitignore file is available in your repo then Bito will also use that to ignore files & folders from indexing process. Both .bitoignore and .gitignore files can work together without any issues.
At present, Bito considers only those .gitignore files that are placed in the project root directory and .bitoignore files that are placed either in <user-home-directory\.bito\localcodesearch>
or <project-root-directory>
Changes to the .bitoignore
file are taken into account at the beginning of the indexing process, not during or after the indexing itself.
Therefore, to implement changes made to the .bitoignore
file, you'll need to delete the index and then restart the indexing by clicking on the "Start Indexing" button shown for the repo folder. You can also follow our step-by-step guides to Start Indexing in Visual Studio Code and Start Indexing in JetBrains IDEs.
Please note that any changes to the .bitoignore or .gitignore file will take a minimum of 3 to 5 minutes to trigger new indexing.
Common .bitoignore Patterns
Understanding these patterns/rules is crucial for effectively managing the files and directories that Bito indexes and excludes in your projects.
Negation !
(exclamation mark)
!
(exclamation mark)When a pattern starts with !
it negates the pattern, meaning it explicitly includes files or directories that would otherwise be ignored. For example, have a look at this sample .bitoignore file:
Here !Engine/Build/BatchFiles/**
pattern includes all files in the Engine/Build/BatchFiles
directory and its subdirectories, even though Engine/**
pattern would ignore them.
Avoid Ambiguous Patterns: Negation patterns can become confusing when they potentially match multiple files. Target specific files or folders rather than using wildcards in negation patterns.
For example, it is better to use patterns like !Engine/Build/BatchFiles/script.bat
instead of !Engine/Build/BatchFiles/**
.bitoignore Examples
Exclude Files/Folders
Exclude Everything Except Specific Files
To exempt a file, ensure that the negation pattern !
appears afterward, thereby overriding any broader exclusions.
Last updated