This is not the first experience of this kind. But it is a sign of things to come where we may begin to, for actual real and practical reasons, come to rely more on "high functioning" LLM models. In this case I used OpenAI's ChatGPT o1 model.
I use neovide to edit code now. I have been driving neovim for code editing in a terminal for a long time, but recently I have switched to neovide because the smooth scrolling, high performance, and other nice features finally pushed me to leave terminal software behind when it comes to editing code.
I started working more with Python software recently and one of my challenges was a byproduct of the (very very good) new trend to manage dependencies for Python projects in venvs (Another overview. Now, instead of going by default to install a software dependency globally in the system and eventually risk getting your system python install into a bloated and bad state, we put only what we need in a well defined place near or in our project and it makes it a lot easier to work with and start over if we screw something up.
Now the problem with this when it comes to the development process is how to tell your editor about these venvs. Because now that we are putting our dependencies in these project specific locations instead of globally, the editor out of the box can only see global python libraries. When I say "see", I'm really talking about how a language server protocol (LSP) server like pyright gets configured and the abilities that it provides to the editor (be that VS Code or neovim or anything else, the dev community has firmly committed to LSP as an interface for this sort of thing by now).
So I needed a way to make it so my editor, which is neovide, can automatically figure out how to inform its Python LSPs to look for deps in any venvs associated with a project when i open its code.
I searched and found many different potential solutions. I think I could have searched farther and wider, but the impression I had was that the most reasonable approach to do this typically would be to activate the venv as usual in the shell and then launch neovim.
But I don't do that anymore. My neovims are all launched by neovide and that's getting launched at the macOS application level, not typically through my shell... I wanted to find a solution at the neovim level that does not rely on shell environment.
In this instance I was lazy and actually just inserted this question in the middle of a home improvement chat I was having with ChatGPT...
The issue I have is that neovide launches the nvims under a separate process chain, so it's not going to work as seamlessly as activating the venv in the shell and then launching neovim. That's how I used to run neovim, but no longer now that I use neovide. I want to say that the most promising path is to add customization to each python LSP that I have set up with Mason to tell them to somehow scan (for each buffer I guess?) for nearby venv dirs and automatically add those locations to the python path? Something like this. I just do not know how to implement it.
o1-2024-12-17 via ChatGPT Plus
If you’re launching Neovim via Neovide, the usual “activate venv, then start nvim” trick won’t help Python LSPs find your virtual environment. Instead, you’ll need to configure each LSP to detect and use the right venv.
Below is a simplified example using pyright with Neovim’s built-in LSP. You’d adapt this to other Python LSPs similarly:
Example: ~/.config/nvim/lua/lsp-config.lua local lspconfig = require("lspconfig")
A small helper function that searches for a local venv folder
For instance, it might check "./.venv" or "./venv" in the project.
local function find_venv(root_dir)
local paths = { "venv", ".venv" }
for _, p in ipairs(paths) do
local full_path = root_dir .. "/" .. p .. "/bin/python"
if vim.fn.executable(full_path) == 1 then
return full_path
end
end
return nil
end
lspconfig.pyright.setup({
on_new_config = function(new_config, root_dir)
local venv_python = find_venv(root_dir)
if venv_python then
new_config.settings.python.pythonPath = venv_python
end
end,
})
How this works:
1. find_venv(root_dir): Scans the project directory for a venv or .venv folder containing Python.
2. on_new_config: Whenever the LSP starts in a new project, it automatically sets the pythonPath to that venv if found.
This way, your Python LSP should pick up the correct virtual environment for each project without relying on a manually activated shell. You can adapt a similar approach for other language servers or use a plugin that supports automatic Python venv detection.
This gave me a lot to go on in the way of research. Not gonna lie, we cannot get very far without validating things like API calls that are brought up out of the blue, because it may very well be completely hallucinated... Turns out this one is a very real one, though, and it would not have been easy for me to zero in on it otherwise.
nvim-lsp doc states
{on_new_config} (function(new_config, new_root_dir)) Function executed after a root directory is detected. This is used to modify the server configuration (including cmd itself). Most commonly, this is used to inject additional arguments into cmd.
If overriding on_new_config, ensure that you read the on_new_config defined in the source file of the default configuration in lspconfig. The original on_new_config snippet for a given server should likely be included in your new override. Some configurations use on_new_config to dynamically set or modify cmd.
Where do we go to check the existing on_new_config for a given lsp server or config? I'm using mason to manage them.
o1-2024-12-17 via ChatGPT Plus
You’ll want to look at nvim-lspconfig’s source files for each LSP server. If you installed nvim-lspconfig (often via Mason or another plugin manager), you can open the server configuration files directly from your local install.
(Depending on how you installed it, the path may vary.)
Inside the corresponding Lua file, you’ll often find something like:
default_config = {
...
on_new_config = function(...)
...
end,
}
That’s the default on_new_config you’ll want to reference. If you override it in your own config, be sure to call or include that original function so you don’t break the server’s default behavior.
Well, it isn't a long story, but to put it short, o1's suggestion works perfectly for me. That's all for this one. Just feels good to have something that can help when google fails and leaves you trying to read between lines that are too far apart.