Infinite loop in generating models from JSONSchema
Describe the bug
I am trying to produce pydantic models from a JSONSchema file I have. When I try to do this, the process never finishes and just accumulates memory without end. I let it run for a while and it ended up taking up 8+GB of memory. The schema itself is a good handful of megabytes with probably over 10,000 discrete components, which could be a problem, but I believe it should stop eventually.
Eventually this stack trace is produced with a RecursionError:
Traceback (most recent call last):
File "C:\Users\Spencer\repos\redacted\venv\lib\site-packages\datamodel_code_generator\__main__.py", line 626, in main
generate(
File "C:\Users\Spencer\repos\redacted\venv\lib\site-packages\datamodel_code_generator\__init__.py", line 384, in generate
results = parser.parse()
File "C:\Users\Spencer\repos\redacted\venv\lib\site-packages\datamodel_code_generator\parser\base.py", line 475, in parse
self.parse_raw()
File "C:\Users\Spencer\repos\redacted\venv\lib\site-packages\datamodel_code_generator\parser\jsonschema.py", line 1270, in parse_raw
self._resolve_unparsed_json_pointer()
File "C:\Users\Spencer\repos\redacted\venv\lib\site-packages\datamodel_code_generator\parser\jsonschema.py", line 1294, in _resolve_unparsed_json_pointer
self._resolve_unparsed_json_pointer()
File "C:\Users\Spencer\repos\redacted\venv\lib\site-packages\datamodel_code_generator\parser\jsonschema.py", line 1294, in _resolve_unparsed_json_pointer
self._resolve_unparsed_json_pointer()
File "C:\Users\Spencer\repos\redacted\venv\lib\site-packages\datamodel_code_generator\parser\jsonschema.py", line 1294, in _resolve_unparsed_json_pointer
self._resolve_unparsed_json_pointer()
[Previous line repeated 957 more times]
File "C:\Users\Spencer\repos\redacted\venv\lib\site-packages\datamodel_code_generator\parser\jsonschema.py", line 1290, in _resolve_unparsed_json_pointer
self.parse_json_pointer(self.raw_obj, reserved_ref, path_parts)
File "C:\Users\Spencer\repos\redacted\venv\lib\site-packages\datamodel_code_generator\parser\jsonschema.py", line 1306, in parse_json_pointer
self.parse_raw_obj(
File "C:\Users\Spencer\repos\redacted\venv\lib\site-packages\datamodel_code_generator\parser\jsonschema.py", line 1213, in parse_raw_obj
self.parse_obj(name, JsonSchemaObject.parse_obj(raw), path)
File "pydantic\main.py", line 526, in pydantic.main.BaseModel.parse_obj
File "C:\Users\Spencer\repos\redacted\venv\lib\site-packages\datamodel_code_generator\parser\jsonschema.py", line 208, in __init__
super().__init__(**data)
File "pydantic\main.py", line 340, in pydantic.main.BaseModel.__init__
File "pydantic\main.py", line 1076, in pydantic.main.validate_model
File "pydantic\fields.py", line 886, in pydantic.fields.ModelField.validate
File "pydantic\fields.py", line 1021, in pydantic.fields.ModelField._validate_mapping_like
File "pydantic\fields.py", line 1094, in pydantic.fields.ModelField._validate_singleton
File "pydantic\fields.py", line 884, in pydantic.fields.ModelField.validate
File "pydantic\fields.py", line 1094, in pydantic.fields.ModelField._validate_singleton
File "pydantic\fields.py", line 884, in pydantic.fields.ModelField.validate
File "pydantic\fields.py", line 1101, in pydantic.fields.ModelField._validate_singleton
File "pydantic\fields.py", line 1148, in pydantic.fields.ModelField._apply_validators
File "pydantic\class_validators.py", line 318, in pydantic.class_validators._generic_validator_basic.lambda13
File "pydantic\main.py", line 711, in pydantic.main.BaseModel.validate
File "C:\Users\Spencer\repos\redacted\venv\lib\site-packages\datamodel_code_generator\parser\jsonschema.py", line 208, in __init__
super().__init__(**data)
File "pydantic\main.py", line 340, in pydantic.main.BaseModel.__init__
File "pydantic\main.py", line 1076, in pydantic.main.validate_model
File "pydantic\fields.py", line 884, in pydantic.fields.ModelField.validate
File "pydantic\fields.py", line 1101, in pydantic.fields.ModelField._validate_singleton
File "pydantic\fields.py", line 1148, in pydantic.fields.ModelField._apply_validators
File "pydantic\class_validators.py", line 318, in pydantic.class_validators._generic_validator_basic.lambda13
File "pydantic\validators.py", line 61, in pydantic.validators.str_validator
RecursionError: maximum recursion depth exceeded while calling a Python object
The schema is too large to put in the issue, but it can be found in this gist.
Used commandline:
$ datamodel-codegen --input problem.json --input-file-type jsonschema --output model.py
Expected behavior
The expectation is that the model generation eventually completes.
Version:
- OS: Windows 10 22H2 (build 19045.2364)
- Python version: 3.10.2
- datamodel-code-generator version: 0.14.1
Additional context
The schema itself was created by dynamically generating pydantic models and dumping model.json_schema(). Not sure if that's relevant, but in my mind I guess it's not out of the realm of possibility that this could matter.
How does funding with Polar work?
Pay now to fund the work behind this issue.
Get updates on progress being made.
Maintainer is rewarded once the issue is completed.
Backer
You're funding impactful open source efforts
Contributor
You want to contribute to this effort
Maintainer
You want to get funding like this too