makePythonPoetryEnvironment #1171

Closed
opened 2023-09-26 19:15:27 +02:00 by danmur97 · 3 comments
danmur97 commented 2023-09-26 19:15:27 +02:00 (Migrated from github.com)

Problem to solve

makePythonPypiEnvironment internally uses poetry to determine the full dependencies tree and outputs a source-able template and generates its own lock format.
But this brings some drawbacks:

  • the dependencies format is transformed from poetry into a custom format https://github.com/fluidattacks/makes/issues/1149
  • the output is incompatible with nix standard python env and pkgs derivations making difficult to integrate https://github.com/fluidattacks/makes/issues/1161
  • pypi envs cannot be merged together into 1 python env. This causes that vscode settings util cannot be used to setup the full env plus brings some python env integration issues (same pkgs with diferent versions enabled)

Intended users

Python developers

Permissions and Security

Proposal

  • Support makePythonPoetryEnvironment that receives a common poetry-lock (solving #1149) outputs a nix python env derivation (like makePyprojectPackage, and solving #1161 and env integration)
  • try https://github.com/nix-community/poetry2nix
  • this util will replace makePythonPypiEnvironment

Test plan

What does success look like, and how can we measure that?

makePythonPypiEnvironment

/label ~"type::feature"

### Problem to solve `makePythonPypiEnvironment` internally uses poetry to determine the full dependencies tree and outputs a source-able template and generates its own lock format. But this brings some drawbacks: - the dependencies format is transformed from poetry into a custom format https://github.com/fluidattacks/makes/issues/1149 - the output is incompatible with nix standard python env and pkgs derivations making difficult to integrate https://github.com/fluidattacks/makes/issues/1161 - pypi envs cannot be merged together into 1 python env. This causes that vscode settings util cannot be used to setup the full env plus brings some python env integration issues (same pkgs with diferent versions enabled) ### Intended users Python developers ### Permissions and Security ### Proposal - Support `makePythonPoetryEnvironment` that receives a common poetry-lock (solving #1149) outputs a nix python env derivation (like `makePyprojectPackage`, and solving #1161 and env integration) - try https://github.com/nix-community/poetry2nix - this util will replace `makePythonPypiEnvironment` ### Test plan ### What does success look like, and how can we measure that? `makePythonPypiEnvironment` ### Links / references /label ~"type::feature"
dsalaza4 commented 2024-01-05 18:32:16 +01:00 (Migrated from github.com)

I will address making makePythonPypiEnvironment use poetry standard files on this issue

I will address making `makePythonPypiEnvironment` use poetry standard files on this issue
jmesa85 commented 2024-01-15 20:52:04 +01:00 (Migrated from github.com)

New builtin makePythonEnvironment generates an error: home directory '/homeless-shelter' exists; please remove it to assure purity of builds without sandboxing on jobs using the arch amd64.

https://us-east-1.console.aws.amazon.com/batch/home?region=us-east-1#jobs/ec2/detail/aa714a44-0ef6-4419-a082-070f76eaeac1

New builtin makePythonEnvironment generates an `error: home directory '/homeless-shelter' exists; please remove it to assure purity of builds without sandboxing` on jobs using the arch amd64. https://us-east-1.console.aws.amazon.com/batch/home?region=us-east-1#jobs/ec2/detail/aa714a44-0ef6-4419-a082-070f76eaeac1
dsalaza4 commented 2024-01-16 18:05:53 +01:00 (Migrated from github.com)

@jmesa85 This is related to Nix not being able to run in sandbox mode on non-privileged containers

@jmesa85 This is related to Nix not being able to run in sandbox mode on non-privileged containers
This discussion has been locked. Commenting is limited to contributors.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
justin/makes#1171
No description provided.