Describe the bug
There appears to be a bug when it comes to local file references; specifically for the paths section. While iterating over the paths
object, there is a piece of code which resolves the references:
datamodel-code-generator/datamodel_code_generator/parser/openapi.py
Lines 572 to 573 in 739b050
The reference resolution here works as expected, but it loses the context of where the reference is from. In other words, if I was pointing to a file at ../../my-file.yaml
then that path may no longer be valid. For example, let's say that we have a simple example as follows:
To Reproduce (1st case)
Folder structure:
/
openapi.yaml
paths/
test.yaml
components/
schemas/
test_object.yaml
openapi.yaml:
openapi: 3.1.0
info:
title: OpenAPI
description: OpenAPI
version: 1.0.0
paths:
/test:
$ref: ./paths/test.yaml
components:
schemas:
test_object:
$ref: ./components/schemas/test_object.yaml
servers:
- url: http://localhost:3000
paths/test.yaml:
get:
responses:
"200":
description: "OK"
content:
application/json:
schema:
$ref: "../components/schemas/test_object.yaml"
components/schemas/test_object.yaml:
type: object
properties:
name:
type: string
Used commandline:
$ datamodel-codegen --input-file-type openapi --input openapi.yaml --output t.py --openapi-scopes schemas paths parameters
Expected behavior
In a normal scenario, since I've specified the paths
and parameters
scopes for openapi I would expect that the library provides Pydantic models as a result. However, what ends up happening is nothing gets created at all. The library doesn't report an error, but I would expect something to be created for this path. Here's what I got:
# generated by datamodel-codegen:
# filename: openapi.yaml
# timestamp: 2023-09-28T00:09:01+00:00
from __future__ import annotations
from typing import Optional
from pydantic import BaseModel
class TestObject(BaseModel):
name: Optional[str] = None
To Reproduce (2nd case)
For the 2nd case, leave everything the exact same, but we're going to change how our path is setup. Rather than referencing a schema directly, we will instead create a new object that has references like so:
paths/test.yaml:
get:
responses:
"200":
description: "OK"
content:
application/json:
schema:
type: object
properties:
data:
$ref: "../components/schemas/test_object.yaml"
Used commandline:
$ datamodel-codegen --input-file-type openapi --input openapi.yaml --output t.py --openapi-scopes schemas paths parameters
Expected behavior
What happens in this case is it attempts to find the ../components/schemas/test_object.yaml
file, but it appends that route onto where the openapi.yaml
file is which makes the first route invalid. It should have started at ./paths
, but it started at ./
.
Redacted my computers file path and swapped it with ...
Traceback (most recent call last):
File ".../datamodel_code_generator/__main__.py", line 388, in main
generate(
File ".../datamodel_code_generator/__init__.py", line 435, in generate
results = parser.parse()
File ".../datamodel_code_generator/parser/base.py", line 1058, in parse
self.parse_raw()
File ".../datamodel_code_generator/parser/openapi.py", line 592, in parse_raw
self.parse_operation(
File ".../datamodel_code_generator/parser/openapi.py", line 522, in parse_operation
self.parse_responses(
File ".../datamodel_code_generator/parser/openapi.py", line 371, in parse_responses
data_types[status_code][content_type] = self.parse_schema(
File ".../datamodel_code_generator/parser/openapi.py", line 326, in parse_schema
self.parse_ref(obj, path)
File ".../datamodel_code_generator/parser/jsonschema.py", line 1498, in parse_ref
self.parse_ref(property_value, path)
File ".../datamodel_code_generator/parser/jsonschema.py", line 1476, in parse_ref
self.resolve_ref(obj.ref)
File ".../datamodel_code_generator/parser/jsonschema.py", line 1466, in resolve_ref
self._get_ref_body(relative_path),
File ".../datamodel_code_generator/parser/jsonschema.py", line 1418, in _get_ref_body
return self._get_ref_body_from_remote(resolved_ref)
File ".../datamodel_code_generator/parser/jsonschema.py", line 1431, in _get_ref_body_from_remote
return self.remote_object_cache.get_or_put(
File ".../datamodel_code_generator/parser/__init__.py", line 28, in get_or_put
value = self[key] = default_factory(key)
File ".../datamodel_code_generator/parser/jsonschema.py", line 1433, in <lambda>
default_factory=lambda _: load_yaml_from_path(full_path, self.encoding),
File ".../datamodel_code_generator/__init__.py", line 51, in load_yaml_from_path
with path.open(encoding=encoding) as f:
File "/opt/homebrew/Cellar/[email protected]/3.9.18/Frameworks/Python.framework/Versions/3.9/lib/python3.9/pathlib.py", line 1252, in open
return io.open(self, mode, buffering, encoding, errors, newline,
File "/opt/homebrew/Cellar/[email protected]/3.9.18/Frameworks/Python.framework/Versions/3.9/lib/python3.9/pathlib.py", line 1120, in _opener
return self._accessor.open(self, flags, mode)
FileNotFoundError: [Errno 2] No such file or directory: './../components/schemas/test_object.yaml'
Version:
Additional context
Again, I believe the issue comes from the fact that for path resolution specifically, the library resolves the reference locally and "thinks" that everything that was resolved is therefore wherever openapi.yaml
is. However, that's not always the case as a path could be referenced under a different directory. Since that context is lost, any references inside of that path file will then be invalid since the starting location has changed.
Pay now to fund the work behind this issue.
Get updates on progress being made.
Maintainer is rewarded once the issue is completed.
You're funding impactful open source efforts
You want to contribute to this effort
You want to get funding like this too