freshcrate
Home > Frameworks > dagster-aws

dagster-aws

Package for AWS-specific Dagster framework solid and resource components.

Description

Package for AWS-specific Dagster framework solid and resource components.

Release History

VersionChangesUrgencyDate
0.29.1Imported from PyPI (0.29.1)Low4/21/2026
1.13.1### New - Added `PipesCompositeMessageReader` (preview) to support multiple concurrent message streams in a single Pipes session. - Added `sensor:`, `schedule:`, and `job:` attribute support to the asset selection syntax (e.g., `sensor:my_sensor`, `job:my_job`). - Added `automation_type:` attribute support to the asset selection syntax, allowing queries like `automation_type:schedule` or `automation_type:sensor`. (Thanks, [@bengotow](https://github.com/bengotow)!) - State-backed integratioHigh4/17/2026
1.13.0### Major Changes Since 1.12.0 - **AI-assisted development**: Released [dagster-io/skills](https://github.com/dagster-io/skills), a collection of Dagster-focused AI skills for coding agents like Claude Code, OpenAI Codex, and others. Expanded `dg api` commands for programmatic inspection of assets, runs, jobs, schedules, and more. - **Partitioned asset checks**: Asset checks can now target specific partitions of an upstream asset, aligning data quality logic with how partitioned data is produceMedium4/9/2026
1.12.22### New - [dg] `dg` projects can now configure `agent_queue` and `image` in `pyproject.toml` under `[tool.dg.project]`, which are included in the generated `dagster_cloud.yaml` when running `dg plus deploy`. - [dg] Added `dg api job list` and `dg api job get` commands for querying job metadata including schedules, sensors, and tags. - [dg] Added `dg api asset-check list`, `dg api asset-check get-executions`, and `dg api asset get-partition-status` commands. - [ui] The asset overview page now shMedium4/2/2026
1.12.21### New - Removed the `setuptools<82` pin from the `dagster` package. - Added support for the `partitions` attribute in asset selection syntax to filter assets by partition definition type (e.g., `partitions:"static"`). - [dagster-spark] Added Spark Declarative Pipeline support in feature preview. - [ui] Added a date range picker to the backfill modal for date-formatted partitions. - [ui] The "Report evaluation" action for asset checks has been moved to a consolidated dropdown menu. ### BugfixMedium3/26/2026
1.12.20### New - Execution context classes (`OpExecutionContext`, `AssetExecutionContext`, `AssetCheckExecutionContext`) now expose a `multi_partition_key` property that returns a `MultiPartitionKey` when the current run is a multi-partition run. - Added `Braze` and `Runpod` kind tags. (Thanks, [@dragos-pop](https://github.com/dragos-pop)!) - [dagster-databricks] The Databricks job run URL is now rendered as a clickable link in the Dagster UI. - [dagster-dbt] Added `DbtCloudComponent` for loading dbt Low3/19/2026
1.12.19### New - Added a "Report Execution" dialog to the asset checks detail view, allowing users to manually record check evaluation results. - Added database pool configuration options (`--db-pool-recycle`, `--db-pool-pre-ping`, and others) to `dg dev` and `dagster dev`. - Added `dg plus config view` command for inspecting the current CLI configuration. - [ui] Updated the Usage dialog and Run timeline to display "Jobless asset materializations" (previously "Ad hoc materializations") with a tooltip Low3/12/2026
1.12.18### New - Added event type filtering to the asset Recent Updates timeline, allowing users to toggle between Materializations and Observations when both event types are present. Filter preference is persisted to local storage. - `psycopg2-binary` has been removed as a dependency from `dagster-postgres`. If you were previously relying on this transitive dependency, you may need to explicitly add `psycopg2-binary` to your project. - [ui] The launchpad UI no longer prefills optional resource deLow3/5/2026
1.12.17### Bugfixes - Fix bug with inclusion of built webapp in `dagster-webserver` that caused the Dagster UI to fail to load.Low2/27/2026
1.12.15### New - Pool names can now be any non-whitespace character, instead of requiring alphanumbeic characters, slashes, and underscores. - [dagster-aws] The `EcsRunLauncher` will now retry a task launch when a RunTask API call fails due to a throttling error in an underlying EC2 API call. - [dagster-tableau] Tableau workbooks and projects can now be filtered using the `workbook_selector` and `project_selector` in `TableauComponent`. ### Bugfixes - [ui] Fixed issue where saved launchpad config waLow2/20/2026
1.12.14### New - `@asset_check` and `AssetCheckSpec` now support a `partitions_def` parameter, allowing checks to execute against specific partitions of their upstream asset rather than the entire contents. If set, the partition definition must match the definition of the targeted asset. - [ui] The "Select all" checkbox has been restored to the Automations list. ### Bugfixes - Fixed performance issue where the partition selector would freeze for 30+ seconds when selecting "All" on assets with large Low2/5/2026
1.12.13### New - [dagster-polytomic] `PolytomicComponent` has been added and can be used to represent your Polytomic bulk sync schemas as external assets in Dagster - [dagster-fivetran] Added warning log when no Fivetran groups are found to help users troubleshoot permission issues.Low1/30/2026
1.12.12### New - `dg plus deploy start` now validates deployment akin to `dagster-cloud ci check`. - [dagster-aws] Added a suite of new components that map to all existing resources in the library. - [dagster-k8s] Increased the maximum version of kubernetes to 35.x.x. - [ui] You can now unpin asset groups that no longer contain any assets. ### Bugfixes - Fixed type errors reported by Pyright's strict mode when using `@asset`, `@multi_asset`, and `@graph_asset` decorators. - Running `dg launch --partLow1/22/2026
1.12.11### New - The `dagster-cloud ci check` command is now marked as deprecated. Use `dg plus deploy start` instead, which now validates configuration during deployment initialization. - [dagster-fivetran] `FivetranWorkspace` now supports a `request_backoff_factor` parameter for enabling exponential backoff on request failures. ### Bugfixes - [dagster-dbt] Fixed an issue where the `exclude` parameter of `@dbt_assets` could be ignored if the selection was too large. - [ui] The Asset Partitions pageLow1/15/2026
1.12.10### Bugfixes - [ui] Fixed an issue introduced in 1.12.9 where the "Catalog" tab in the Dagster UI sometimes failed to display any assets.Low1/9/2026
1.12.9### New - The core `dagster` package (and most libraries) are now compatible with Python 3.14. - Added support for using python version 3.13 when running `dg plus deploy`. - `dg plus login` now supports a `region` flag for eu-based users: `dg plus login --region eu`. - Updated the `bulk_actions` table `body` column from `Text` to `LongText` for Mysql storage. To take advantage of this migration run `dagster instance migrate`. (Thanks, [@jenkoian](https://github.com/jenkoian)!) - Updated the `asLow1/8/2026
1.12.8### New - `dg plus deploy` commands now support Python 3.13 and Python 3.14. ### Bugfixes - Fixed an issue where the Dagster Helm chart and Dagster+ agent helm chart could no longer deploy using Helm without adding the `--skip-schema-validation` flag to the Helm deploy command. Thanks [@kang8](https://github.com/kang8)!Low12/29/2025
1.12.7### New - Optimized performance of calculating partition keys for time window partitions with exclusions. - `timedelta` and `datetime` are now available via the `datetime` context when rendering components (Thanks, [@stevenayers](https://github.com/stevenayers)!) - `FreshnessPolicy` is now available via the `dg` context when rendering components. (Thanks, [@stevenayers](https://github.com/stevenayers)!) - Assets may now be annotated with up to 10 kinds (limit was previously 3). - Arbitrary resoLow12/19/2025
1.12.6### New - All CLI commands under `dagster project` have been removed. `create-dagster` should be used instead. - [ui] Added a new Partitions facet to the Asset Lineage Graph. - [ui] More details are now displayed for `SINCE` conditions in evaluation tables for automation conditions. - [dagster-dbt] Added dbt cloud logs to stdout after the run completes in dbt cloud. - [dagster-tableau] Improved resilience when fetching Tableau workspace data. The integration now skips individual workbooks that Low12/11/2025
1.12.5## New - Increased the version of NextJS used by the Dagster webserver and the `dg docs serve` command to `15.5.7`. While these applications are unaffected by https://nextjs.org/blog/CVE-2025-66478 due to not using React 19, this upgrade ensures that dagster packages will not be flagged for that CVE by vulnerability scanners.Low12/9/2025
1.12.4### New - CI workflows for Gitlab projects can now be scaffolded using `dg plus deploy configure`. - "/" characters are now allowed in concurrency pool names. - Pod wait timeout for K8sPipeClient can now be specified (Thanks, [@abhinavDhulipala](https://github.com/abhinavDhulipala)!) - New `kind` tag icon for Zendesk (Thanks, [@kporter13](https://github.com/kporter13)!) - [dagster-tableau] Added `enable_embedded_datasource_refresh` and `enable_published_datsource_refresh` options to the `TableaLow12/4/2025
1.12.3### New - Updated the cursoring logic of `AutomationCondition.since()`/`AutomationCondition.newly_true()` to make them retain access to their stored data in a wider range of scenarios where the underlying condition structure is changed. - Added a `--use-active-venv` method to a variety of `dg` commands. (Thanks, [@cmpadden](https://github.com/cmpadden)!) - The `build_defs_at_path` and `load_component_at_path` methods on the `ComponentLoadContext` class have been renamed to `build_defs` and `loaLow11/20/2025
1.12.2### New - Dagster has dropped support for Python 3.9 (due to end of life). The minimum supported Python version is now 3.10. This is a breaking change originally intended for the 1.12.0 release. - Added `dg api schedule {list,get}` command. - Added new `dg plus deploy configure` CLI group that generates all the files necessary to get an existing project ready for deployment via Dagster+. - [ui] A new sidebar in the Asset catalog (behind a feature flag) provides a hierarchical view of your assetLow11/13/2025
1.12.1### New - Migrate SqlComponent to Pydantic v2 model_config pattern (Thanks, [@LPauzies](https://github.com/LPauzies)!) - Make `dg api` commands visible in `dg --help` output. - Make `dg plus` commands visible in the `dg --help` output. - Add Dremio kind icon. (Thanks, [@maxfirman](https://github.com/maxfirman)!) - The github actions scaffolded by `dg scaffold github-actions` now include commands to refresh state for `StateBackedComponents`. - Run worker health check will now tag runs with theirLow11/7/2025
1.12.0## Major changes since 1.11.0 (core) / 0.27.0 (libraries) ### **UI** - **Refreshed UI:** The UI has been redesigned and streamlined to make it easier to find common utilities quickly. Navigation elements have been moved from a header to a collapsible sidebar to help highlight important workflows and provide more visual space throughout the product. ### Components - **Components GA:** The Components framework and the `dg` CLI are now marked as GA (previously Release Candidate). The APIs are fLow10/30/2025
1.11.16### New - The proxy GRPC server heartbeat TTL can now be configured with the DAGSTER_GRPC_PROXY_HEARTBEAT_TTL_SECONDS env var (default remains 30 seconds). ### Bugfixes - Fixed an issue introduced in dagster 1.11.15 where code locations that previously loaded would sometimes fail to load with a `gRPC Error code: RESOURCE_EXHAUSTED` error. - Fixed an issue where defining a repository using a dictionary of job definitions with a key that did not match the name of the job would work when runningLow10/23/2025
1.11.15### New - All sequences are now supported in `AssetKey.with_prefix`. (Thanks, [@aksestok](https://github.com/aksestok)!) - [ui] Introduce new navigation, with main navigation items previously in top navigation now in a collapsible left nav. - [ui] Improve loading performance of Runs page. - [dagster-databricks] Add support for `notebook_task` in `PipesDatabricksClient`. (Thanks, [@SoerenStahlmann](https://github.com/SoerenStahlmann)!) ### Bugfixes - Fixed an issue where `fetch_row_counts` andLow10/16/2025
1.11.14### New - `UnionTypes` (e.g. `Foo | Bar`) are now supported in `ConfigurableResources`. (Thanks, [@DominikHallab](https://github.com/DominikHallab)!) - Added an `output_metadata` parameter to `build_output_context` (Thanks, [@zyd14](https://github.com/zyd14)!) - `ResolvedAssetSpec` and related resolvers now support setting the `freshness_policy` field. - `dagster-dbt project prepare-and-package --components .` will no longer attempt to load components outside of `DbtProjectComponent`, preventinLow10/9/2025
1.11.13### New - [dagster-deltalake,dagster-deltalake-polars] BREAKING CHANGE - we now support `deltalake>=1.0.0` for `dagster-deltalake` and `dagster-deltalake-polars` and we will no longer support `deltalake<1.0.0` moving forward. End user APIs remain the same for both libraries. - [dagster-databricks] Spark Python and Python Wheel tasks are now supported in `PipesDatabricksServerlessClient`. - [dagster-dbt] `dagster-dbt project prepare-and-package --components .` will no longer attempt to load compLow10/2/2025
1.11.12### New - [ui] Allow searching across code locations with `*` wildcard in selection inputs for jobs and automations. - [ui] Added `AutomationCondition.all_new_executed_with_tags`, which allows automation conditions to be filtered to partitions that have been materialized since the last tick from runs with certain tags. This condition can be used to require or prevent certain run tags from triggering downstream declarative automation conditions. ### Bugfixes - In `dagster==1.11.1`, `partitioneLow9/25/2025
1.11.11### New - `anthropic`, `mcp`, and `claude-code-sdk` dependencies of `dagster-dg-cli` are now under a separate `ai` extra, allowing `dagster-dg-cli` to be installed without these dependencies. - Added `AutomationCondition.all_new_updates_have_run_tags` and `AutomationCondition.any_new_update_has_run_tags`, which allows automation conditions to be filtered to partitions that have been materialized since the last tick from runs with certain tags. This condition can be used to require or prevent ceLow9/18/2025
1.11.10### New - Added `inline-component` command to the publicly available scaffold commands in the Dagster CLI. - Added a new `require_upstream_step_success` config param to all executors. If `{"step_dependency_config": {"require_upstream_step_success": False}}` is set, this will allow downstream steps to execute immediately after all required upstream outputs have finished, even if the upstream step has not completed in its entirety yet. This can be useful particularly in cases where there are largLow9/11/2025
1.11.9### New - Subclasses of `Resolved` now support fields of type `dict[str, T]`. - [ui] Added a new 'arrow' icon to the set of supported kind tags (thanks [@aleewen](https://github.com/aleewen)!) ### Bugfixes - Launching a backfill of a non-subsettable multi-asset without including every asset will now raise a clear error at backfill submission time, instead of failing with a confusing error after the backfill has started. - Fixed an issue where passing in an empty list to the `assetKeys` argumeLow9/4/2025
1.11.8### New - A param `exclusions` was added to time window partition definitions to support custom calendars. - The `dagster` library now supports `protobuf==6.x` - [dg] `dg scaffold defs --help` now shows descriptions for subcommands. - [dg] A new `dg check toml` command has been added to validate your TOML configuration files. - [dagster-databricks] The `DatabricksAssetBundleComponent` has been added in preview. Databricks tasks can now be represented as assets and submitted via Dagster. - [dagsLow8/28/2025
1.11.7### New - `dg` will now report multiple detected errors in a configuration file instead of failing on the first detected error. - It is now possible to supply run config when launching an asset backfill. - Updated the root URL to display the Overview/Timeline view for locations with schedules/automations, but no jobs (thanks [@dschafer](https://github.com/dschafer)!) - Added `tzdata` as a dependency to `dagster`, to ensure that declaring timezones like `US/Central` work in all environments. - [Low8/22/2025
1.11.6### New - Allow explicit git `platform` selection in `link_code_references_to_git`, thanks [@chazmo03](https://github.com/chazmo03)! ### Bugfixes - Fixed issue causing `AutomationCondition.replace` to not update built-in sub-conditions that did not have an explicit label, thanks [@dschafer](https://github.com/dschafer)! - Fixed an issue where assets were considered stubs if they were a stub in any code location. - Projects using components no longer cause "job definitions changed while uploadLow8/14/2025
1.11.5### New - Static functions on classes decorated with `@template_var` can now optionally accept a `ComponentLoadContext` argument. - [dg] A MCP server is available to expose `dg` CLI capabilities to MCP clients. See the `dg mcp` CLI group for details. - [dagster-dbt] The `dagster-dbt` package no longer has a dependency on `dbt-core`. - [dagster-dbt][preview] Users of the dbt Fusion CLI can now use the `dagster-dbt` package to run dbt commands with no changes to their existing dagster code. This Low8/7/2025
1.11.4### New - Schedules now support specifying a subset of asset checks to execute in a `RunRequest`. - [dg] A new `docs integrations` cli is available for viewing an index of available integrations. - [ui] Jobs can now be filtered with a selection syntax. - [dagster-tableau] Dashboards containing hidden sheets are now correctly linked to upstream data sources. - [dagster-tableau] Tableau sheets and dashboards now produce observation events instead of materialization events when using `refresh_and_Low7/31/2025
1.11.3### New - Introduced `AssetExecutionContext.load_asset_value`, which enables loading asset values from the IO manager dynamically rather than requiring asset values be loaded as parameters to the asset function. For example: ```python @dg.asset(deps=[the_asset]) def the_downstream_asset(context: dg.AssetExecutionContext): return context.load_asset_value(dg.AssetKey("the_asset")) ``` - Expose asset_selection parameter for `submit_job_execution` function in DagsterGraphQLClient, thankLow7/24/2025
1.11.2### New - The `dagster` package now supports Python 3.13. - [dagster-tableau] Tableau assets can now be subsetted and materialized individually. [#31078](https://github.com/dagster-io/dagster/pull/31078) - [dagster-snowflake-polars] The new `dagster-snowflake-polars` package adds a `SnowflakePolarsIOManager` that can be used to read and write Snowflake tables using Polars. ### Bugfixes - [ui] Fixed some cases where strings would be incorrectly middle-truncated. ### Documentation - [dbt] FixLow7/10/2025
1.11.1### New - `dagster definitions validate` will now raise an exception if there are invalid partition mappings between any assets in your asset graph (for example, an upstream and downstream asset with time-based partitions definitions using different timezones). - Performance improvements for run dequeuing when there are many queued runs using pools. - [ui] For times in the last two days, Dagster UI now shows e.g. "47 hours ago" instead of "2 days ago." - [ui] Asset checks now show whether they Low7/3/2025
1.11.0## Major changes since 1.10.0 (core) / 0.26.0 (libraries) ### Components — Configurable, reusable building blocks for data pipelines (now stable) Components, first released as Preview in 1.10.6, have reached Release Candidate status. APIs are stable, fully documented, and are the recommended, production-ready defaults for new Dagster projects. - **Simplified interface:** A short YAML block in `defs.yaml`, or a lightweight `Component` subclass in Python, lets you spin up arbitrary Dagster defiLow6/26/2025
1.10.21### New - [dagster-tableau] The `tableau_assets` decorator is now available to create the asset definitions of all materializable assets in a given Tableau workspace. These assets can be subsetted and materialized using the `TableauCloudWorkspace.refresh_and_poll` method. - [dagster-tableau] The deprecated `get_*_asset_key` methods on the `DagsterTableauTranslator` class have been removed. - [ui] Show tags for a Job on the sidebar of the Job page. [#30728](https://github.com/dagster-io/dagster/Low6/20/2025
1.10.20### New - The `@asset` decorator now supports a `hooks` argument to allow for functions to be executed on asset success / failure (thanks @brunobbaraujo)! - The log message produced when an asset check is evaluated now includes its pass / fail state. - The `@multi_asset_check` decorator now supports the `pool` argument. - [dagster-dbt] The `DagsterDbtTranslator` class now has a `get_asset_check_spec` method which can be overridden to customize the `AssetCheckSpecs` that are produced for each inLow6/13/2025
1.10.19### New - The database table used by the `DbIOManager` is now configurable via `"table"` output (asset) metadata key [#30310](https://github.com/dagster-io/oss/pull/30310) - Changed default settings for backfill daemon to `use_threads=True`, `num_workers=4`. Thanks [@HynekBlaha](https://github.com/HynekBlaha)! - A new function `build_asset_check_context` can be used to build asset check contexts for direct invocation. - Changed `Definitions.get_all_asset_specs` to only return assets directly paLow6/5/2025
1.10.18### New - `BackfillPolicy` is now marked as generally available (GA). - Optimized the order of the `@asset` decorator overloads to make custom wrappers around the asset decorator easier. Thanks [@jonathanunderwood](https://github.com/jonathanunderwood)! - [dagster-slack] Added `get_client()` to SlackResource. ### Bugfixes - `Definitions` and `AssetDefinition` will now warn if they get different `AssetSpec`s with the same key. This will become an exception in 1.11. - Functions that load all deLow5/29/2025
1.10.17### Bugfixes - Fixed an issue where an error was displayed in the UI while viewing run logs - [dagster-dbt] Fixed an issue occurring when using dbt selection arguments with a dbt project using semantic models.Low5/23/2025
1.10.16### New - `typing_extensions` is now pinned to `>=4.11.0` instead of `>=4.10.0`. - [ui] Viewing an automation condition evaluation now automatically expands the set of applicable sub-conditions. - [ui] Added the ability to navigate from an automation condition evaluation to upstream automation condition evaluations. - [ui] Added an asset graph node facet for viewing automation conditions and the most recent evaluation. This can be enabled in the user settings via a feature flag (`Enable facetedLow5/22/2025
1.10.15### New - Added a config section to `dagster.yaml` to enable submitting backfill runs in a threadpool. - Expanded definition time validation for partition mappings to avoid runtime errors querying asset status. - [ui][beta] You can now re-execute a run that targeted a multi-asset from the point of asset failure instead of step failure, meaning only assets that failed or were skipped will be re-executed. To enable this option, turn on the `Enable retries from asset failure` feature flag in your Low5/15/2025
1.10.14### New - [dagster-tableau] Refined Tableau integration for API 3.25 or greater. - [dagster-tableau] Data sources with extracts can now be materialized in Tableau assets created with `build_tableau_materializable_assets_definition`. - [ui] Added kinds tag for treasuredata. - [ui] Add Supabase kind icon. ### Bugfixes - Fixed a bug which could cause an error when calling `MultiPartitionsDefinition.has_partition_key()` on invalid keys. - Fixed a bug where the default multiprocess executor would Low5/8/2025
1.10.13### New - If an unselected asset check is executed during a run, the system will now warn instead of throwing a hard error. - When evaluating `AutomationCondition.any_deps_match` or `AutomationCondition.all_dep_match` with an allow / ignore specified, an error will no longer be produced if the provided asset selection references an asset key that does not exist. - Added the ability to restrict the list of ports that `dagster dev` is allowed to use to open subprocesses when running on WindowLow5/1/2025

Dependencies & License Audit

Loading dependencies...

Similar Packages

pre-commitA framework for managing and maintaining multi-language pre-commit hooks.v4.6.0
azure-core-tracing-opentelemetryMicrosoft Azure Azure Core OpenTelemetry plugin Library for Pythonazure-template_0.1.0b6187637
spdx-toolsSPDX parser and tools.0.8.5
lacesDjango components that know how to render themselves.0.1.2
django-tasksA backport of Django's built in Tasks framework0.12.0