Preface

We want to implement a feature that allows the various features of Biome to tap into information regarding other files. At the moment, features like lint rules don't have access to information that belongs to other files. Some examples are exported bindings, exported types, types of modules, etc.

An entity we could call Project (the name is not set in stone) should provide this kind of information.

Design

Following, I will list some design aspects of the feature, which we can discuss. Don't take everything as set in stone, but assimilate it and then debate it. I will try to do my best to outline the current archicture as best as I can.

First, the architecture should follow this principle:

• Build for generic clients. Don’t assume that a terminal will only consume output using ANSI codes. Use abstractions that could be generalized for viewing in an IDE, browser, or other environments.

Playing actors

At the moment, we have the current actors at play:

• The FileSystem trait. A generic trait meant to crawl the file system and evaluate if certain paths (directories, files, symbolic links, etc.) should be checked. Two types implement this trait OsFileSystem and MemoryFileSystem. The FileSystem doens't know which files should be handled. The logic that checks if those paths can be handled is implemented elsewhere via another trait called TraversalContext. As you can see, we are going elsewhere. OsFileSystem, however, has an interner for paths, but it isn't related to the paths checked.
• The WorkspaceServer can be seen as our query compiler. The end consumer (e.g. CLI or LSP) has control over the compiler and tells it what to do. The WorkspaceServer has some characteristics:
  • It contains the content of each file.
  • It contains the CST of each file.
  • Doesn't which files should be checked. The end consumer knows it.
  • The concept of ProjectData is already in use. It knows that a client might have different projects with different settings:
    

    biome/crates/biome_service/src/settings.rs

    Lines 52 to 57 in eafc62e

    	pub struct WorkspaceSettings {
    	/// The data of the projects
    	data: WorkspaceData<ProjectData>,
    	/// The ID of the current project.
    	current_project: ProjectKey,
    	}

The FileSystem and the WorkspaceServer rarely interact with each other. The only moment when they is for just one operation: fs.open_file -> workspace.open_file

Possible data flow

I envision the Project to be part of the WorkspaceServer. If so, Project must be Send, Sync and RefUnwindSafe. This means that we need to choose data that meets those criteria.

Fortunately, a module graph can be built with petgraph, which implements these traits.

A Project shouldn't duplicate the information that are already present in the WorkspaceServer, e.g. file content, CST.

Where we should strive

Here are some ideas that we should consider while implementing the architecture:

• try to compute data only when requested;
• don't overdo it, try to compute only the data requested;
• think about queries, where a consumer asks for data, and the provider delivers them (and ideally caches them);

CLI

From a CLI perspective, a way to build a Project and its graph is by doing two passes of the file system.

One pass to collect all files. A second pass to check the files. After the first pass it could be possible build the Project we require.

The second pass is more challenging because, between the first pass and the second pass, we don't have an entity that will tell the WorkspaceServer which files should be checked. Should we create one? Should we extend FileSystem to tell us? Let's discuss it

LSP

From the LSP perspective, it's easier to initialise a new Project, but it will be more challenging to update it. Every time a file is updated/created/deleted, the graph should be updated.

References

Here, Jeroen Engels, creator of elm-review, explains how he does multi-file analysis inside the linter: https://jfmengels.net/multi-file-analysis/ .
However, let's take into consideration that we have different constraints and requirements.

Tasks

I haven't talked about dependencies. Dependencies will be part of the graph, but they can be added later one, as we need a manifest to collect them (which we already have).

• Create a type Project inside biome_project
• Create and model the project graph
• Breakdown the CLI traversal into two passes. It doesn't require the project to be broken down.
• Create and model an API to update the graph based on file paths and imports
• Create and model APIs to CRUD the module graph

(Feel free to add more tasks if you think they are needed)