Skip to content

[Community] init Iceberg support #3094

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Oct 1, 2025
Merged

[Community] init Iceberg support #3094

merged 1 commit into from
Oct 1, 2025

Conversation

GuillaumeDSM
Copy link
Member

remaining:

  • make async

@GuillaumeDSM GuillaumeDSM self-assigned this Sep 30, 2025
@GuillaumeDSM GuillaumeDSM changed the title WIP [Community] init Iceberg support [Community] init Iceberg support Oct 1, 2025
@GuillaumeDSM GuillaumeDSM marked this pull request as ready for review October 1, 2025 08:59
@GuillaumeDSM GuillaumeDSM requested a review from Herklos as a code owner October 1, 2025 08:59
Herklos
Herklos reviewed Oct 1, 2025
View reviewed changes
Copy link
Member

@Herklos Herklos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work !!

additional_tests/historical_backend_tests/test_iceberg_candles.py
assert 0 < min_time < max_time < time.time()


async def test_fetch_candles_history(iceberg_client):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

octobot/community/history_backend/iceberg_historical_backend_client.py Outdated


class TableNames(enum.Enum):
OHLCV_HISTORY = "ohlcv_history6"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😄

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we use an env var here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rigtht, done

GuillaumeDSM
GuillaumeDSM commented Oct 1, 2025
View reviewed changes
octobot/community/history_backend/iceberg_historical_backend_client.py

async def open(self):
try:
self._executor = concurrent.futures.ThreadPoolExecutor(max_workers=_MAX_EXECUTOR_WORKERS)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added this to use a local executor that is limited and closed when we don't need it anymore

Herklos
Herklos approved these changes Oct 1, 2025
View reviewed changes
Copy link
Member

@Herklos Herklos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💯

octobot/constants.py
ICEBERG_CATALOG_NAMESPACE = os.getenv("ICEBERG_CATALOG_NAMESPACE")
ICEBERG_CATALOG_NAME = os.getenv("ICEBERG_CATALOG_NAME")
ICEBERG_CATALOG_WAREHOUSE = os.getenv("ICEBERG_CATALOG_WAREHOUSE")
ICEBERG_OHLCV_HISTORY_TABLE = os.getenv("ICEBERG_OHLCV_HISTORY_TABLE")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@GuillaumeDSM GuillaumeDSM enabled auto-merge (rebase) October 1, 2025 09:22
@GuillaumeDSM GuillaumeDSM merged commit e5e3697 into dev Oct 1, 2025
11 checks passed
@GuillaumeDSM GuillaumeDSM deleted the iceberg branch October 1, 2025 09:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Herklos Herklos Herklos approved these changes

Assignees

@GuillaumeDSM GuillaumeDSM

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@GuillaumeDSM @Herklos