Repository navigation
Pydantic to suit the Python library used in your examples?????? #30
Description
Activity
@jondricek if you solve this, then you are a major guru, thanks for taking notice of my request
@brettforbes The ATT&CK Data Model was born out of a need for type hinting and data validation in other MITRE-maintained projects. Naturally, ATT&CK-oriented software tends to focus heavily on either the presentation layer (e.g., the website, navigator, etc.) or data analysis. The majority of our tooling focuses on the presentation layer:
- The TAXII 2.1 server is implemented in TypeScript.
- The ATT&CK Workbench, which includes a Node.js backend, is implemented in JavaScript and TypeScript.
- The ATT&CK website will eventually be re-written using a TypeScript oriented framework like Next.js, React, Angular, etc.
- The ATT&CK Navigator is implemented in TypeScript
To date, the only active, Python-native project that our team supports is the mitreattack-python library. We plan to add type hinting and data validation to that project in the future.
We conducted an exhaustive evaluation before settling on Zod, comparing it to other frameworks including attrs and Pydantic. You can actually see traces of attrs and Pydantic schemas in the commit history of this project. We recognize the need for Python type hinting and validation. However, given that the vast majority of our tooling is JS/TS focused, Zod was the path of least resistance to codify the ATT&CK specification while providing the largest benefit across our project ecosystem.
We recognize this isn't ideal for all community members. Originally, we wanted to implement a domain-specific language (DSL) that could codify the ATT&CK spec in an intuitive format and emit to multiple languages (similar to Google Protobuf). That's our long-term vision, but we don't currently have the resources to implement our own DSL. Until then, we've opted to stick with a framework that benefits the most projects under the ATT&CK software umbrella.
Note that Zod does support emitting to JSON schema. It's possible to emit JSON schemas and either use them directly or generate Pydantic schemas from them. We haven't tested this approach ourselves yet, but some quick searches indicate it's a popular topic.
Reacted by brettforbesTo add to what @seansica wrote, the Python stix2 library (PyPI link) is not maintained by our team. And the original concept did revolve around writing JSON schemas that would be the official source of truth that either our TypeScript or Python libraries would be able to reference. But while that's how it started, the current implementation is what you find here in this repository.
We haven't forgotten or abandoned the mitreattack-python library. It will continue to be maintained - and it is our intention to have it use the schemas that come from the ATT&CK Data Model repo here in the long run. We just need to make sure it isn't a manual or overly burdensome process on our end to keep them in sync when the ATT&CK specification changes.
Reacted by brettforbesThank you guys for giving me the back-story, I really appreciate it.
I am currently extending it to suit OCA and Attack Flow, and we hope to upgrade the whole system at the end of the year. We now use Pydantic quite extensively, hence my question.
However, this Python Library is used within our low-code system (almost finished), and we plan to host the Navigator as an iFrame, in our CMS (Content Management System) pages. So the good news is that we are benefiting from your zod templates, by using that front end.
We currently create our own templates based on your USAGE.md file, and have found this to be a satisfactory approach.
Given your feedback we will continue on with this approach, and not try to integrate with your own Zod system. Thanks a lot for your explanation, and especially the backstory so I could understand.
Cheers
Reacted by Jared Ondricek
Hmm, the decision to write out your schemas in zod is difficult to understand.
What use are these schemas, apart from for a user interfaces? Surely, the vast majority of Attack users use Python, so why zod???
Is there an intention to produce a Pydantic library to suit interactions with your existing code base, such as:
You guys were absolute gurus when you wrote the Stix2 Python Library for OASIS, it is a beautiful job. Very clever coding work, but why is Python being isolated and TypeScript introduced?
What are the Python users of your Data and library meant to do?
Can you advise?