""" # `queries` Package The `queries` package is designed for loading and executing queries against databases. It provides functionality to load queries from files or strings and execute them asynchronously or synchronously. Additionally, it includes utilities for comparing dataframes. ## queries.query.Query This module is responsible for: - Loading queries from files, directories or strings. - Executing queries against databases. - Handling both **synchronous** and **asynchronous** query execution. > We recommend using the `Queries` class instead of the `Query` class, as it > provides additional functionality for managing multiple queries. ## queries.query.Queries This module is responsible for: - Managing multiple queries. - Executing multiple queries asynchronously or synchronously. - Reordering queries based on a provided list of query names. - Indexing and slicing queries. - Loading queries from files or strings. - loading queries from directories. - Executing queries against databases. - Handling both **synchronous** and **asynchronous** query execution. - Loading multiple environments and executing them against databases. > To load queries from a file or from a string you need to follow the > specific structure defined in queries.query.get_format ## queries.compare.get_differences This method is responsible for: - Comparing two dataframes to identify differences. ## Installation To install the `queries` package, add the following line to your `pyproject.toml` file (edit paths as needed): ```toml [tool.poetry.dependencies] python = "^3.12" queries = {path = "", develop = true} ``` After you specify the package in your `pyproject.toml` file, you can install it by running: ```bash poetry install ``` --- Clickhouse and Postgres connections need environment variables to be set. Setting the environment variables can be done: + **implicitly** by loading environment variables into the shell or + **explicitly** by setting them in the code using the configuration file. ## **implicitly** The package expects the following environment variables: - for clickhouse connection **clickhouse**: - `CLICKHOUSE_HOST` - `CLICKHOUSE_PORT` - `CLICKHOUSE_USER` - `CLICKHOUSE_PASSWORD` - `CLICKHOUSE_DATABASE` - for postgres connection **postgres**: - `POSTGRES_HOST` - `POSTGRES_PORT` - `POSTGRES_USER` - `POSTGRES_PASSWORD` - `POSTGRES_DATABASE` ## **explicitly** You just need to follow the following structure ``: ```json { "": { "CLICKHOUSE_HOST": "", "CLICKHOUSE_PORT": "", "CLICKHOUSE_USER": "", "CLICKHOUSE_PASSWORD": "", "CLICKHOUSE_DATABASE": "", }, "": { "POSTGRES_HOST": "", "POSTGRES_PORT": "", "POSTGRES_USER": "", "POSTGRES_PASSWORD": "", "POSTGRES_DATABASE": "" } } ``` After you save the file, you can load the environments using the `Queries` object: ```python from queries.query import Queries queries = Queries( source='', env_config_file='' ) # run on specific environment for example queries.run_queries(env_name='') ``` Variables in the configuration files have higher priority than environment variables. If you set the same variable in both the configuration file and the environment, the configuration file will be used. > Don't forget that when calling a query, you need to specify the environment > name if you have multiple environments loaded (if not set, environment > variables will be used). ## Comparing Dataframes If you want to use methods for comparing dataframes, you can use the `compare` module. It provides a method for comparing two dataframes and identifying differences between them. ```python import pandas as pd from queries.compare import get_differences a = pd.DataFrame({'a': [1, 2, 3, 4], 'b': [5, 6, 7, 8]}) b = pd.DataFrame({'a': [1, 2, 3, 5], 'b': [5, 6, 7, 8]}) print(get_differences(a, b)) ``` """