Skip to content

How to mark a model field as "non-persisted", i.e. to not be pushed to the DB? #147

Closed

Description

@JLHasson

First Check

  • I added a very descriptive title to this issue.
  • I used the GitHub search to find a similar issue and didn't find it.
  • I searched the SQLModel documentation, with the integrated search.
  • I already searched in Google "How to X in SQLModel" and didn't find any information.
  • I already read and followed all the tutorial in the docs and didn't find an answer.
  • I already checked if it is not related to SQLModel but to Pydantic.
  • I already checked if it is not related to SQLModel but to SQLAlchemy.

Commit to Help

  • I commit to help with one of those options 👆

Example Code

from typing import Optional
from pydantic.fields import PrivateAttr

from sqlmodel import Field, Session, SQLModel, create_engine, select


class Hero(SQLModel, table=True):
    id: Optional[int] = Field(default=None, primary_key=True)
    _name: str = PrivateAttr()  # This field is not committed to the db
    secret_name: str


sqlite_file_name = "database.db"
sqlite_url = f"sqlite:///{sqlite_file_name}"

engine = create_engine(sqlite_url, echo=True)

if __name__ == "__main__":
    SQLModel.metadata.create_all(engine)

    hero_1 = Hero(secret_name="Dive Wilson")
    hero_1._name = "hello"
    print(hero_1)  # Hero will have a _hello attribute

    with Session(engine) as session:
        session.add(hero_1)

        session.commit()
    with Session(engine) as session:
        statement = select(Hero)
        results = session.exec(statement)
        for hero in results:
            print(hero)  # Hero will not have a _hello attribute (when read from the db)

Description

It is unclear how to add fields to a model that should not be written to the DB. I have achieved this by prepending the attribute name with an underscore as shown in the code sample above. I decided to achieve it that way based on how Pydantic handles these private attributes (https://pydantic-docs.helpmanual.io/usage/models/#automatically-excluded-attributes).

If there is a better way to do this I'd love to know. Alternatively I'd love to update the documentation if this is the correct way. I took a look at the file structure a bit but wasn't entirely sure where the best place to add it was, so I decided to open an issue instead of a PR.

Operating System

macOS

Operating System Details

No response

SQLModel Version

0.0.4

Python Version

Python 3.9.5

Additional Context

No response

Activity

  1. adjaunzemis commented on Oct 27, 2021

    @adjaunzemis

    It looks like you're working through the tutorials. I think the HeroBase vs Hero vs HeroRead vs... construct using inheritance in the later sections on "FastAPI and Pydantic" might be what you're looking for:
    https://sqlmodel.tiangolo.com/tutorial/fastapi/multiple-models/
    Basically, only the class with table=true will define the table columns. You can inherit from this class to add other fields that won't show up in the table.

  2. JLHasson commented on Oct 28, 2021

    @JLHasson
    Author

    @adjaunzemis thanks for the response. Looking through that page again it looks like I can get to what I want, but IMO it's messy. Here's what I've done:

    class HeroBase(SQLModel):
        name: str
    
    
    class HeroTable(HeroBase, table=True):
        id: Optional[int] = Field(default=None, primary_key=True)
    
    
    # I want to use `Hero` everywhere in my codebase, I should never instantiate
    # an instance of `HeroTable` or `HeroBase`
    class Hero(HeroBase):
        secret_name: str = None  # need default here to fill when read from DB
    
    
    with Session(engine) as session:
        hero_1 = Hero(secret_name="Dive Wilson", name="Lance")
    
        # Now I need to cast every `Hero` instance to `HeroTable` before
        # I can write to the DB, this is messy
        hero_db = HeroTable.from_orm(hero_1)
        session.add(hero_db)
    
        # I can also only read `HeroTable` instances
        statement = select(HeroTable)
        results = session.exec(statement)
            for hero in results:
                # And need to cast it to `Hero` every time
                hero_obj = Hero.parse_obj(hero)
                print(hero_obj)  # name='Lance' secret_name=None
    

    This works but IMO it's messy. From the docs:

    Only inherit from data models, don't inherit from table models.
    It will help you avoid confusion, and there won't be any reason for you to need to inherit from a table model.
    If you feel like you need to inherit from a table model, then instead create a base class that is only a data model and has all those fields, like HeroBase.
    And then inherit from that base class that is only a data model for any other data model and for the table model.

    It would be much nicer if I could just do

    class HeroBase(SQLModel, table=True):
        id: Optional[int] = Field(default=None, primary_key=True)
        name: str
    
    class Hero(HeroBase):
        secret_name: str = None
    

    And then just use Hero everywhere, and only have the values in HeroBase (the class with table=True) committed to the DB.

    Unless there is a better way to do the casting that I've done above I think I will just keep using PrivateAttr along with @property decorators to keep values on the same class and just know that private attributes aren't written to the DB.

    class Hero(HeroBase):
        id: Optional[int] = Field(default=None, primary_key=True)
        name: str
        _secret_name: str = PrivateAttr(default=None)  # this attribute isn't written to the DB
    
        @property
        def secret_name(self):
            return self._secret_name
    
        def set_secret_name(self, secret_name):
            self._secret_name = secret_name
    

    Of course, this comes with it's own set of problems 🙄 pydantic/pydantic#1577

    Also, as far as I can tell SQLModel doesn't properly set the default=None value from pydantic.PrivateAttr on table classes, may open a separate issue for that one.

  3. JLHasson commented on Oct 28, 2021

    @JLHasson
    Author

    Closing because the comment above technically solves the issue, but IMO it could be improved

  4. JLHasson commented on Oct 28, 2021

    @JLHasson
    Author

    Opened new issue for the PrivateAttr issue: #149

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    questionFurther information is requested

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions