List database tables with SQLAlchemy

I want to implement a function that gives information about all the tables (and their column names) that are present in a database (not only those created with SQLAlchemy). While reading the documentation it seems to me that this is done via reflection but I didn't manage to get something working. Any suggestions or examples on how to do this?

Solution 1:

start with an engine:

from sqlalchemy import create_engine
engine = create_engine("postgresql://u:p@host/database")

quick path to all table /column names, use an inspector:

from sqlalchemy import inspect
inspector = inspect(engine)

for table_name in inspector.get_table_names():
   for column in inspector.get_columns(table_name):
       print("Column: %s" % column['name'])

docs: http://docs.sqlalchemy.org/en/rel_0_9/core/reflection.html?highlight=inspector#fine-grained-reflection-with-inspector

alternatively, use MetaData / Tables:

from sqlalchemy import MetaData
m = MetaData()
m.reflect(engine)
for table in m.tables.values():
    print(table.name)
    for column in table.c:
        print(column.name)

docs: http://docs.sqlalchemy.org/en/rel_0_9/core/reflection.html#reflecting-all-tables-at-once

Solution 2:

Hey I created a small module that helps easily reflecting all tables in a database you connect to with SQLAlchemy, give it a look: EZAlchemy

from EZAlchemy.ezalchemy import EZAlchemy

DB = EZAlchemy(
    db_user='username',
    db_password='pezzword',
    db_hostname='127.0.0.1',
    db_database='mydatabase',
    d_n_d='mysql'   # stands for dialect+driver
)

# this function loads all tables in the database to the class instance DB
DB.connect()

# List all associations to DB, you will see all the tables in that database
dir(DB)

Solution 3:

First set up the sqlalchemy engine.

from sqlalchemy import create_engine, inspect, text
from sqlalchemy.engine import url

connect_url = url.URL(
    'oracle',
    username='db_username',
    password='db_password',
    host='db_host',
    port='db_port',
    query=dict(service_name='db_service_name'))

engine = create_engine(connect_url)

try:
    engine.connect()
except Exception as error:
    print(error)
    return

Like others have mentioned, you can use the inspect method to get the table names.

But in my case, the list of tables returned by the inspect method was incomplete.

So, I found out another way to find table names by using pure SQL queries in sqlalchemy.

query = text("SELECT table_name FROM all_tables where owner = '%s'"%str('db_username'))

table_name_data = self.session.execute(query).fetchall()

Just for sake of completeness of answer, here's the code to fetch table names by inspect method (if it works good in your case).

inspector = inspect(engine)
table_names = inspector.get_table_names()

Solution 4:

I'm proposing another solution as I was not satisfied by any of the previous in the case of postgres which uses schemas. I hacked this solution together by looking into the pandas source code.

from sqlalchemy import MetaData, create_engine
from typing import List

def list_tables(pg_uri: str, schema: str) -> List[str]:
    with create_engine(pg_uri).connect() as conn:
        meta = MetaData(conn, schema=schema)
        meta.reflect(views=True)
        return list(meta.tables.keys())

In order to get a list of all tables in your schema, you need to form your postgres database uri pg_uri (e.g. "postgresql://u:p@host/database" as in the zzzeek's answer) as well as the schema's name schema. So if we use the example uri as well as the typical schema public we would get all the tables and views with:

list_tables("postgresql://u:p@host/database", "public")