![]() It's developed by volunteers and it cannot reallyy provide that high of an availablity and self -maintenance. ![]() In the rare even of hardware failure it might happen that things got broken and yes in this case it might happen some manual intervention will be required.Īirflow is not 99.999 availablity kind of software. Shutting down any application abruptly might always lead to inconsistencies and problem and you should avoid doing it. : Instance '' has been deleted, or its row is otherwise not present.įirst of all, you should gracefully restart Airlfow components when you restart them. Return strategy._load_for_state(state, passive)įile "/home/airflow/.local/lib/python3.10/site-packages/sqlalchemy/orm/strategies.py", line 530, in _load_for_state Value = self._fire_loader_callables(state, key, passive)įile "/home/airflow/.local/lib/python3.10/site-packages/sqlalchemy/orm/attributes.py", line 976, in _fire_loader_callablesįile "/home/airflow/.local/lib/python3.10/site-packages/sqlalchemy/orm/strategies.py", line 561, in _call_ To_reset = _adopt_task_instances(tis_to_reset_or_adopt)įile "/home/airflow/.local/lib/python3.10/site-packages/airflow/executors/celery_executor.py", line 503, in try_adopt_task_instancesįile "/home/airflow/.local/lib/python3.10/site-packages/sqlalchemy/orm/attributes.py", line 482, in _get_įile "/home/airflow/.local/lib/python3.10/site-packages/sqlalchemy/orm/attributes.py", line 942, in get ![]() Return func(*args, session=session, **kwargs)įile "/home/airflow/.local/lib/python3.10/site-packages/airflow/jobs/scheduler_job.py", line 1413, in adopt_or_reset_orphaned_tasksįor attempt in run_with_db_retries(logger=self.log):įile "/home/airflow/.local/lib/python3.10/site-packages/tenacity/_init_.py", line 347, in _iter_įile "/home/airflow/.local/lib/python3.10/site-packages/tenacity/_init_.py", line 314, in iterįile "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 451, in resultįile "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 403, in _get_resultįile "/home/airflow/.local/lib/python3.10/site-packages/airflow/jobs/scheduler_job.py", line 1458, in adopt_or_reset_orphaned_tasks Other 3rd-party Helm chart Deployment detailsįile "/home/airflow/.local/bin/airflow", line 8, in įile "/home/airflow/.local/lib/python3.10/site-packages/airflow/_main_.py", line 48, in mainįile "/home/airflow/.local/lib/python3.10/site-packages/airflow/cli/cli_parser.py", line 52, in commandįile "/home/airflow/.local/lib/python3.10/site-packages/airflow/utils/cli.py", line 108, in wrapperįile "/home/airflow/.local/lib/python3.10/site-packages/airflow/cli/commands/scheduler_command.py", line 73, in schedulerįile "/home/airflow/.local/lib/python3.10/site-packages/airflow/cli/commands/scheduler_command.py", line 43, in _run_scheduler_jobįile "/home/airflow/.local/lib/python3.10/site-packages/airflow/jobs/base_job.py", line 258, in runįile "/home/airflow/.local/lib/python3.10/site-packages/airflow/jobs/scheduler_job.py", line 759, in _executeįile "/home/airflow/.local/lib/python3.10/site-packages/airflow/jobs/scheduler_job.py", line 840, in _run_scheduler_loopįile "/home/airflow/.local/lib/python3.10/site-packages/airflow/utils/session.py", line 75, in wrapper Not sure if reproducible outside our system Operating Systemĭebian GNU/Linux 11 (bullseye) Versions of Apache Airflow ProvidersĪpache-airflow-providers-cncf-kubernetes=5.2.2Īpache-airflow-providers-common-sql=1.3.4Īpache-airflow-providers-elasticsearch=4.4.0Īpache-airflow-providers-hashicorp=3.3.0Īpache-airflow-providers-snowflake=4.0.4Īpache-airflow-providers-ssh=3.5.0 Deployment We encounter issue with every restart/redeploy. Restart airflow-scheduler/redeploy whole airflow while tasks are running (are being processed by Scheduler/Workers) Scheduler should properly shutdown gracefuly in given time and properly restart afterward without raising ObjectDeletedError How to reproduce Airflow Scheduler started to have issue when being restarted (either manually or forcefuly) - some task instances are stuck in running/queued state after restart and Scheduler somehow misses reference to them (or fails to readopt them) resulting in critical error about TaskInstance missing.Įrror requires manual intervention into airflow database (Setting stuck tasks manually to failed state) What you think should happen instead
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |