Last active
June 24, 2021 11:24
-
-
Save silashansen/5ae2d8c3c3a9877265276d7ff48164ef to your computer and use it in GitHub Desktop.
Clickhouse log
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| 2021.06.24 08:39:52.882235 [ 49925 ] {} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202106_330750_330750_0, total 54 rows starting from the beginning of the part | |
| 2021.06.24 08:39:52.893411 [ 49925 ] {} <Debug> analytics.events_local (8fcee572-2b74-42f2-8fce-e5722b74c2f2) (MergerMutator): Merge sorted 2913 rows, containing 46 columns (46 merged, 0 gathered) in 0.012852513 sec., 226648.28271327174 rows/sec., 94.10 MiB/sec. | |
| 2021.06.24 08:39:52.914845 [ 49925 ] {} <Trace> analytics.events_local (8fcee572-2b74-42f2-8fce-e5722b74c2f2): Renaming temporary part tmp_merge_202106_330565_330750_17 to 202106_330565_330750_17. | |
| 2021.06.24 08:39:52.914934 [ 49925 ] {} <Trace> analytics.events_local (8fcee572-2b74-42f2-8fce-e5722b74c2f2) (MergerMutator): Merged 5 parts: from 202106_330565_330743_16 to 202106_330750_330750_0 | |
| 2021.06.24 08:39:52.918118 [ 49925 ] {} <Debug> MemoryTracker: Peak memory usage: 12.40 MiB. | |
| 2021.06.24 08:39:53.272905 [ 49929 ] {} <Trace> system.metric_log (22746af7-8e26-4b0f-a274-6af78e26db0f): Found 1 old parts to remove. | |
| 2021.06.24 08:39:53.272944 [ 49929 ] {} <Debug> system.metric_log (22746af7-8e26-4b0f-a274-6af78e26db0f): Removing part from filesystem 202106_51987_51987_0 | |
| 2021.06.24 08:39:53.337866 [ 49925 ] {} <Trace> system.query_thread_log (1e30a702-a0b8-4399-9e30-a702a0b86399): Found 1 old parts to remove. | |
| 2021.06.24 08:39:53.337915 [ 49925 ] {} <Debug> system.query_thread_log (1e30a702-a0b8-4399-9e30-a702a0b86399): Removing part from filesystem 202106_51559_51559_0 | |
| 2021.06.24 08:39:53.403314 [ 49929 ] {} <Trace> system.trace_log (6716feed-ee28-476d-a716-feedee28476d): Found 1 old parts to remove. | |
| 2021.06.24 08:39:53.403355 [ 49929 ] {} <Debug> system.trace_log (6716feed-ee28-476d-a716-feedee28476d): Removing part from filesystem 202106_26786_26786_0 | |
| 2021.06.24 08:39:53.810999 [ 49884 ] {} <Trace> HTTPHandler-factory: HTTP Request for HTTPHandler-factory. Method: POST, Address: 172.31.18.125:35650, User-Agent: (none), Length: 48, Content Type: text/sql, Transfer Encoding: identity, X-Forwarded-For: (none) | |
| 2021.06.24 08:39:53.811086 [ 49884 ] {} <Trace> DynamicQueryHandler: Request URI: /?enable_http_compression=true&default_format=RowBinaryWithNamesAndTypes&database=analytics | |
| 2021.06.24 08:39:53.811363 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> executeQuery: (from 172.31.18.125:35650, user: cludo, using production parser) SELECT * FROM events LIMIT 0 | |
| 2021.06.24 08:39:53.812039 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Trace> ContextAccess (cludo): Access granted: SELECT(id, event_type, date, customer_id, engine_id, query, display_width_px, display_height_px, user_agent, referrer_url, referrer_title, browser_language, ip_address, page_number, host_name, session_id, query_session_id, query_raw, device_type, geo_continent_name, geo_city_name, geo_country_iso_code, geo_region_name, geo_latitude, geo_longitude, click_link_url, click_link_index, click_clicked_object_id, click_title, click_query_id, click_context, query_type, query_total_banners, query_total_documents, query_response_time_ms, query_actual_query, query_quick_link_id, query_results_per_page, query_environment, query_request_type, query_banners, query_billable, application_name, application_version, http_status_code, aws_region) ON analytics.events | |
| 2021.06.24 08:39:53.812696 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Trace> ContextAccess (cludo): Access granted: SELECT(id, event_type, date, customer_id, engine_id, query, display_width_px, display_height_px, user_agent, referrer_url, referrer_title, browser_language, ip_address, page_number, host_name, session_id, query_session_id, query_raw, device_type, geo_continent_name, geo_city_name, geo_country_iso_code, geo_region_name, geo_latitude, geo_longitude, click_link_url, click_link_index, click_clicked_object_id, click_title, click_query_id, click_context, query_type, query_total_banners, query_total_documents, query_response_time_ms, query_actual_query, query_quick_link_id, query_results_per_page, query_environment, query_request_type, query_banners, query_billable, application_name, application_version, http_status_code, aws_region) ON analytics.events | |
| 2021.06.24 08:39:53.813310 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Trace> ContextAccess (cludo): Access granted: SELECT(id, event_type, date, customer_id, engine_id, query, display_width_px, display_height_px, user_agent, referrer_url, referrer_title, browser_language, ip_address, page_number, host_name, session_id, query_session_id, query_raw, device_type, geo_continent_name, geo_city_name, geo_country_iso_code, geo_region_name, geo_latitude, geo_longitude, click_link_url, click_link_index, click_clicked_object_id, click_title, click_query_id, click_context, query_type, query_total_banners, query_total_documents, query_response_time_ms, query_actual_query, query_quick_link_id, query_results_per_page, query_environment, query_request_type, query_banners, query_billable, application_name, application_version, http_status_code, aws_region) ON analytics.events_local | |
| 2021.06.24 08:39:53.813606 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> analytics.events_local (8fcee572-2b74-42f2-8fce-e5722b74c2f2) (SelectExecutor): Key condition: unknown | |
| 2021.06.24 08:39:53.813940 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> analytics.events_local (8fcee572-2b74-42f2-8fce-e5722b74c2f2) (SelectExecutor): MinMax index condition: unknown | |
| 2021.06.24 08:39:53.814358 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> analytics.events_local (8fcee572-2b74-42f2-8fce-e5722b74c2f2) (SelectExecutor): Selected 353/353 parts by partition key, 353 parts by primary key, 237717/237717 marks by primary key, 237717 marks to read from 353 ranges | |
| 2021.06.24 08:39:53.814545 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Trace> InterpreterSelectQuery: FetchColumns -> Complete | |
| 2021.06.24 08:39:53.814935 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Trace> InterpreterSelectQuery: Complete -> Complete | |
| 2021.06.24 08:39:53.815250 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 197001_0_0_0_3, approx. 304286 rows starting from 0 | |
| 2021.06.24 08:39:53.815379 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 197001_1_1_0_3, approx. 573440 rows starting from 0 | |
| 2021.06.24 08:39:53.815496 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 197001_2_2_0_3, approx. 356461 rows starting from 0 | |
| 2021.06.24 08:39:53.815617 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201506_0_0_0_1, approx. 414 rows starting from 0 | |
| 2021.06.24 08:39:53.815732 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201507_0_0_0_1, approx. 33195 rows starting from 0 | |
| 2021.06.24 08:39:53.815846 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201508_0_0_0_1, approx. 51793 rows starting from 0 | |
| 2021.06.24 08:39:53.815958 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201509_0_0_0_1, approx. 62205 rows starting from 0 | |
| 2021.06.24 08:39:53.816070 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201510_0_0_0_1, approx. 70681 rows starting from 0 | |
| 2021.06.24 08:39:53.816190 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201511_0_0_0_2, approx. 102361 rows starting from 0 | |
| 2021.06.24 08:39:53.816328 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201511_1_1_0_2, approx. 2083 rows starting from 0 | |
| 2021.06.24 08:39:53.816444 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201512_0_0_0_1, approx. 102113 rows starting from 0 | |
| 2021.06.24 08:39:53.816557 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201601_0_0_0_1, approx. 177717 rows starting from 0 | |
| 2021.06.24 08:39:53.816670 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201602_0_0_0_1, approx. 174400 rows starting from 0 | |
| 2021.06.24 08:39:53.816780 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201603_0_0_0_1, approx. 191476 rows starting from 0 | |
| 2021.06.24 08:39:53.816905 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201604_0_0_0_2, approx. 224207 rows starting from 0 | |
| 2021.06.24 08:39:53.817022 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201604_1_1_0_2, approx. 4831 rows starting from 0 | |
| 2021.06.24 08:39:53.817132 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201605_0_0_0_1, approx. 274460 rows starting from 0 | |
| 2021.06.24 08:39:53.817245 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201606_0_0_0_2, approx. 302384 rows starting from 0 | |
| 2021.06.24 08:39:53.817363 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201606_1_1_0_2, approx. 215874 rows starting from 0 | |
| 2021.06.24 08:39:53.817476 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201607_0_0_0_2, approx. 410811 rows starting from 0 | |
| 2021.06.24 08:39:53.817589 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201607_1_1_0_2, approx. 184266 rows starting from 0 | |
| 2021.06.24 08:39:53.817701 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201608_0_0_0_2, approx. 543210 rows starting from 0 | |
| 2021.06.24 08:39:53.817818 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201608_1_1_0_2, approx. 363254 rows starting from 0 | |
| 2021.06.24 08:39:53.817930 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201609_0_0_0_3, approx. 197630 rows starting from 0 | |
| 2021.06.24 08:39:53.818044 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201609_1_1_0_3, approx. 575041 rows starting from 0 | |
| 2021.06.24 08:39:53.818157 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201609_2_2_0_3, approx. 159488 rows starting from 0 | |
| 2021.06.24 08:39:53.818266 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201610_0_0_0_3, approx. 402120 rows starting from 0 | |
| 2021.06.24 08:39:53.818378 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201610_1_1_0_3, approx. 435347 rows starting from 0 | |
| 2021.06.24 08:39:53.818497 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201610_2_2_0_3, approx. 2752 rows starting from 0 | |
| 2021.06.24 08:39:53.818610 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201611_0_0_0_3, approx. 617391 rows starting from 0 | |
| 2021.06.24 08:39:53.818718 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201611_1_1_0_3, approx. 423790 rows starting from 0 | |
| 2021.06.24 08:39:53.818821 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201611_2_2_0_3, approx. 5248 rows starting from 0 | |
| 2021.06.24 08:39:53.818927 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201612_0_0_0_3, approx. 532480 rows starting from 0 | |
| 2021.06.24 08:39:53.819032 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201612_1_1_0_3, approx. 442368 rows starting from 0 | |
| 2021.06.24 08:39:53.819133 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201612_2_2_0_3, approx. 240929 rows starting from 0 | |
| 2021.06.24 08:39:53.819245 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201701_0_0_0_5, approx. 557056 rows starting from 0 | |
| 2021.06.24 08:39:53.819347 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201701_1_1_0_5, approx. 448356 rows starting from 0 | |
| 2021.06.24 08:39:53.819453 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201701_2_2_0_5, approx. 577038 rows starting from 0 | |
| 2021.06.24 08:39:53.819555 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201701_3_3_0_5, approx. 527687 rows starting from 0 | |
| 2021.06.24 08:39:53.819658 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201701_4_4_0_5, approx. 219716 rows starting from 0 | |
| 2021.06.24 08:39:53.819763 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201702_0_0_0_5, approx. 685046 rows starting from 0 | |
| 2021.06.24 08:39:53.819869 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201702_1_1_0_5, approx. 773654 rows starting from 0 | |
| 2021.06.24 08:39:53.819976 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201702_2_2_0_5, approx. 753664 rows starting from 0 | |
| 2021.06.24 08:39:53.820077 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201702_3_3_0_5, approx. 589824 rows starting from 0 | |
| 2021.06.24 08:39:53.820182 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201702_4_4_0_5, approx. 241671 rows starting from 0 | |
| 2021.06.24 08:39:53.820295 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201703_0_0_0_6, approx. 679936 rows starting from 0 | |
| 2021.06.24 08:39:53.820405 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201703_1_1_0_6, approx. 606208 rows starting from 0 | |
| 2021.06.24 08:39:53.820507 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201703_2_2_0_6, approx. 1024000 rows starting from 0 | |
| 2021.06.24 08:39:53.820611 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201703_3_3_0_6, approx. 516096 rows starting from 0 | |
| 2021.06.24 08:39:53.820713 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201703_4_4_0_6, approx. 735953 rows starting from 0 | |
| 2021.06.24 08:39:53.820823 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201703_5_5_0_6, approx. 204007 rows starting from 0 | |
| 2021.06.24 08:39:53.820925 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201704_0_5_1_6, approx. 3317105 rows starting from 0 | |
| 2021.06.24 08:39:53.821031 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201705_0_5_1_11, approx. 3787725 rows starting from 0 | |
| 2021.06.24 08:39:53.821134 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201705_6_6_0_11, approx. 562936 rows starting from 0 | |
| 2021.06.24 08:39:53.821238 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201705_7_7_0_11, approx. 573440 rows starting from 0 | |
| 2021.06.24 08:39:53.821342 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201705_8_8_0_11, approx. 649493 rows starting from 0 | |
| 2021.06.24 08:39:53.821444 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201705_9_9_0_11, approx. 744334 rows starting from 0 | |
| 2021.06.24 08:39:53.821557 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201705_10_10_0_11, approx. 116339 rows starting from 0 | |
| 2021.06.24 08:39:53.821660 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201706_0_7_1_15, approx. 4580164 rows starting from 0 | |
| 2021.06.24 08:39:53.821765 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201706_8_8_0_15, approx. 231720 rows starting from 0 | |
| 2021.06.24 08:39:53.821874 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201706_9_9_0_15, approx. 917504 rows starting from 0 | |
| 2021.06.24 08:39:53.821976 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201706_10_10_0_15, approx. 688128 rows starting from 0 | |
| 2021.06.24 08:39:53.822080 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201706_11_11_0_15, approx. 129291 rows starting from 0 | |
| 2021.06.24 08:39:53.822182 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201706_12_12_0_15, approx. 851968 rows starting from 0 | |
| 2021.06.24 08:39:53.822290 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201706_13_13_0_15, approx. 1058091 rows starting from 0 | |
| 2021.06.24 08:39:53.822403 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201706_14_14_0_15, approx. 342283 rows starting from 0 | |
| 2021.06.24 08:39:53.822505 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201707_0_6_1_13, approx. 4381991 rows starting from 0 | |
| 2021.06.24 08:39:53.822610 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201707_7_7_0_13, approx. 470486 rows starting from 0 | |
| 2021.06.24 08:39:53.822716 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201707_8_8_0_13, approx. 942080 rows starting from 0 | |
| 2021.06.24 08:39:53.822822 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201707_9_9_0_13, approx. 366592 rows starting from 0 | |
| 2021.06.24 08:39:53.822924 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201707_10_10_0_13, approx. 794624 rows starting from 0 | |
| 2021.06.24 08:39:53.823025 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201707_11_11_0_13, approx. 742685 rows starting from 0 | |
| 2021.06.24 08:39:53.823134 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201707_12_12_0_13, approx. 202135 rows starting from 0 | |
| 2021.06.24 08:39:53.823240 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201708_0_7_1_16, approx. 5710344 rows starting from 0 | |
| 2021.06.24 08:39:53.823344 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201708_8_14_1_16, approx. 5553441 rows starting from 0 | |
| 2021.06.24 08:39:53.823446 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201708_15_15_0_16, approx. 32950 rows starting from 0 | |
| 2021.06.24 08:39:53.823548 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201709_0_7_1_19, approx. 6053883 rows starting from 0 | |
| 2021.06.24 08:39:53.823649 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201709_8_8_0_19, approx. 1197814 rows starting from 0 | |
| 2021.06.24 08:39:53.823760 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201709_9_14_1_19, approx. 3555499 rows starting from 0 | |
| 2021.06.24 08:39:53.823864 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201709_15_15_0_19, approx. 589824 rows starting from 0 | |
| 2021.06.24 08:39:53.823969 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201709_16_16_0_19, approx. 1169150 rows starting from 0 | |
| 2021.06.24 08:39:53.824070 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201709_17_17_0_19, approx. 1124219 rows starting from 0 | |
| 2021.06.24 08:39:53.824191 [ 49884 ] {3904fb93-536a-4132-941f-6440bc9845a0} <Debug> MergeTreeSelectProcessor: Reading 1 ranges from part 201709_18_18_0_19, approx. 365049 rows starting from 0 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| 18. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0x103c36bf in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x103c6e49 in /usr/bin/clickhouse | |
| 20. DB::HTTPServerConnection::run() @ 0x1044acd0 in /usr/bin/clickhouse | |
| 21. Poco::Net::TCPServerConnection::start() @ 0x12a3fd4f in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerDispatcher::run() @ 0x12a417da in /usr/bin/clickhouse | |
| 23. Poco::PooledThread::run() @ 0x12b7ab39 in /usr/bin/clickhouse | |
| 24. Poco::ThreadImpl::runnableEntry(void*) @ 0x12b76b2a in /usr/bin/clickhouse | |
| 25. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 26. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| 2021.06.24 08:39:25.398211 [ 49884 ] {83a67ece-7422-4fbc-a4af-06588afbbc9a} <Error> DynamicQueryHandler: Code: 242, e.displayText() = DB::Exception: Table is in readonly mode (zookeeper path: /clickhouse/analytics/tables/01/events_local), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8b6cbba in /usr/bin/clickhouse | |
| 1. DB::Exception::Exception<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x8c0b8a3 in /usr/bin/clickhouse | |
| 2. DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0xff30579 in /usr/bin/clickhouse | |
| 3. DB::StorageReplicatedMergeTree::write(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>) @ 0xff3068a in /usr/bin/clickhouse | |
| 4. DB::PushingToViewsBlockOutputStream::PushingToViewsBlockOutputStream(std::__1::shared_ptr<DB::IStorage> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::IAST> const&, bool) @ 0xf8b85da in /usr/bin/clickhouse | |
| 5. DB::InterpreterInsertQuery::execute() @ 0xf8b2e2b in /usr/bin/clickhouse | |
| 6. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0x100a5ef5 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0x100a2aa9 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0x100a1cd4 in /usr/bin/clickhouse | |
| 9. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf8b9472 in /usr/bin/clickhouse | |
| 10. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf8c39ab in /usr/bin/clickhouse | |
| 11. DB::SquashingBlockOutputStream::finalize() @ 0xf8c402c in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf8c40c9 in /usr/bin/clickhouse | |
| 13. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xf332d1d in /usr/bin/clickhouse | |
| 14. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf8b6279 in /usr/bin/clickhouse | |
| 15. DB::IBlockInputStream::read() @ 0xf30f452 in /usr/bin/clickhouse | |
| 16. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xf332f76 in /usr/bin/clickhouse | |
| 17. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xfc0eaa5 in /usr/bin/clickhouse | |
| 18. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0x103c36bf in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x103c6e49 in /usr/bin/clickhouse | |
| 20. DB::HTTPServerConnection::run() @ 0x1044acd0 in /usr/bin/clickhouse | |
| 21. Poco::Net::TCPServerConnection::start() @ 0x12a3fd4f in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerDispatcher::run() @ 0x12a417da in /usr/bin/clickhouse | |
| 23. Poco::PooledThread::run() @ 0x12b7ab39 in /usr/bin/clickhouse | |
| 24. Poco::ThreadImpl::runnableEntry(void*) @ 0x12b76b2a in /usr/bin/clickhouse | |
| 25. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 26. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.6.5.37 (official build)) | |
| 2021.06.24 08:39:25.405212 [ 49884 ] {2e328776-1e80-40ce-9a8a-160bcc35115e} <Warning> ClusterProxy::SelectStreamFactory: Local replica of shard 1 is stale (delay: 1624523965s.) | |
| 2021.06.24 08:39:25.543621 [ 49884 ] {21b94e33-7499-41f0-9696-034d2b794c99} <Error> executeQuery: Code: 242, e.displayText() = DB::Exception: Table is in readonly mode (zookeeper path: /clickhouse/analytics/tables/01/events_local) (version 21.6.5.37 (official build)) (from 172.31.18.125:35324) (in query: INSERT INTO events (`id`, `event_type`, `date`, `customer_id`, `engine_id`, `query`, `display_width_px`, `display_height_px`, `user_agent`, `referrer_url`, `referrer_title`, `browser_language`, `ip_address`, `page_number`, `host_name`, `session_id`, `query_session_id`, `query_raw`, `device_type`, `geo_continent_name`, `geo_city_name`, `geo_country_iso_code`, `geo_region_name`, `geo_latitude`, `geo_longitude`, `click_link_url`, `click_link_index`, `click_clicked_object_id`, `click_title`, `click_query_id`, `click_context`, `query_type`, `query_total_banners`, `query_total_documents`, `query_response_time_ms`, `query_actual_query`, `query_quick_link_id`, `query_results_per_page`, `query_environment`, `query_request_type`, `query_banners`, `query_billable`, `application_name`, `application_version`, `http_status_code`, `aws_region`) FORMAT RowBinary ), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8b6cbba in /usr/bin/clickhouse | |
| 1. DB::Exception::Exception<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x8c0b8a3 in /usr/bin/clickhouse | |
| 2. DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0xff30579 in /usr/bin/clickhouse | |
| 3. DB::StorageReplicatedMergeTree::write(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>) @ 0xff3068a in /usr/bin/clickhouse | |
| 4. DB::PushingToViewsBlockOutputStream::PushingToViewsBlockOutputStream(std::__1::shared_ptr<DB::IStorage> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::IAST> const&, bool) @ 0xf8b85da in /usr/bin/clickhouse | |
| 5. DB::InterpreterInsertQuery::execute() @ 0xf8b2e2b in /usr/bin/clickhouse | |
| 6. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0x100a5ef5 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0x100a2aa9 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0x100a1cd4 in /usr/bin/clickhouse | |
| 9. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf8b9472 in /usr/bin/clickhouse | |
| 10. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf8c39ab in /usr/bin/clickhouse | |
| 11. DB::SquashingBlockOutputStream::finalize() @ 0xf8c402c in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf8c40c9 in /usr/bin/clickhouse | |
| 13. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xf332d1d in /usr/bin/clickhouse | |
| 14. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf8b6279 in /usr/bin/clickhouse | |
| 15. DB::IBlockInputStream::read() @ 0xf30f452 in /usr/bin/clickhouse | |
| 16. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xf332f76 in /usr/bin/clickhouse | |
| 17. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xfc0eaa5 in /usr/bin/clickhouse | |
| 18. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0x103c36bf in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x103c6e49 in /usr/bin/clickhouse | |
| 20. DB::HTTPServerConnection::run() @ 0x1044acd0 in /usr/bin/clickhouse | |
| 21. Poco::Net::TCPServerConnection::start() @ 0x12a3fd4f in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerDispatcher::run() @ 0x12a417da in /usr/bin/clickhouse | |
| 23. Poco::PooledThread::run() @ 0x12b7ab39 in /usr/bin/clickhouse | |
| 24. Poco::ThreadImpl::runnableEntry(void*) @ 0x12b76b2a in /usr/bin/clickhouse | |
| 25. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 26. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| 2021.06.24 08:39:25.543805 [ 49884 ] {21b94e33-7499-41f0-9696-034d2b794c99} <Error> DynamicQueryHandler: Code: 242, e.displayText() = DB::Exception: Table is in readonly mode (zookeeper path: /clickhouse/analytics/tables/01/events_local), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8b6cbba in /usr/bin/clickhouse | |
| 1. DB::Exception::Exception<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x8c0b8a3 in /usr/bin/clickhouse | |
| 2. DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0xff30579 in /usr/bin/clickhouse | |
| 3. DB::StorageReplicatedMergeTree::write(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>) @ 0xff3068a in /usr/bin/clickhouse | |
| 4. DB::PushingToViewsBlockOutputStream::PushingToViewsBlockOutputStream(std::__1::shared_ptr<DB::IStorage> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::IAST> const&, bool) @ 0xf8b85da in /usr/bin/clickhouse | |
| 5. DB::InterpreterInsertQuery::execute() @ 0xf8b2e2b in /usr/bin/clickhouse | |
| 6. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0x100a5ef5 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0x100a2aa9 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0x100a1cd4 in /usr/bin/clickhouse | |
| 9. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf8b9472 in /usr/bin/clickhouse | |
| 10. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf8c39ab in /usr/bin/clickhouse | |
| 11. DB::SquashingBlockOutputStream::finalize() @ 0xf8c402c in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf8c40c9 in /usr/bin/clickhouse | |
| 13. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xf332d1d in /usr/bin/clickhouse | |
| 14. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf8b6279 in /usr/bin/clickhouse | |
| 15. DB::IBlockInputStream::read() @ 0xf30f452 in /usr/bin/clickhouse | |
| 16. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xf332f76 in /usr/bin/clickhouse | |
| 17. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xfc0eaa5 in /usr/bin/clickhouse | |
| 18. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0x103c36bf in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x103c6e49 in /usr/bin/clickhouse | |
| 20. DB::HTTPServerConnection::run() @ 0x1044acd0 in /usr/bin/clickhouse | |
| 21. Poco::Net::TCPServerConnection::start() @ 0x12a3fd4f in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerDispatcher::run() @ 0x12a417da in /usr/bin/clickhouse | |
| 23. Poco::PooledThread::run() @ 0x12b7ab39 in /usr/bin/clickhouse | |
| 24. Poco::ThreadImpl::runnableEntry(void*) @ 0x12b76b2a in /usr/bin/clickhouse | |
| 25. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 26. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.6.5.37 (official build)) | |
| 2021.06.24 08:39:25.552110 [ 49884 ] {97da60b3-5481-4d6c-9a26-38227a03547b} <Warning> ClusterProxy::SelectStreamFactory: Local replica of shard 1 is stale (delay: 1624523965s.) | |
| 2021.06.24 08:39:26.576657 [ 49884 ] {70f3496c-924b-470a-9613-2ee00fca0e69} <Error> executeQuery: Code: 242, e.displayText() = DB::Exception: Table is in readonly mode (zookeeper path: /clickhouse/analytics/tables/01/events_local) (version 21.6.5.37 (official build)) (from 172.31.18.125:35336) (in query: INSERT INTO events (`id`, `event_type`, `date`, `customer_id`, `engine_id`, `query`, `display_width_px`, `display_height_px`, `user_agent`, `referrer_url`, `referrer_title`, `browser_language`, `ip_address`, `page_number`, `host_name`, `session_id`, `query_session_id`, `query_raw`, `device_type`, `geo_continent_name`, `geo_city_name`, `geo_country_iso_code`, `geo_region_name`, `geo_latitude`, `geo_longitude`, `click_link_url`, `click_link_index`, `click_clicked_object_id`, `click_title`, `click_query_id`, `click_context`, `query_type`, `query_total_banners`, `query_total_documents`, `query_response_time_ms`, `query_actual_query`, `query_quick_link_id`, `query_results_per_page`, `query_environment`, `query_request_type`, `query_banners`, `query_billable`, `application_name`, `application_version`, `http_status_code`, `aws_region`) FORMAT RowBinary ), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8b6cbba in /usr/bin/clickhouse | |
| 1. DB::Exception::Exception<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x8c0b8a3 in /usr/bin/clickhouse | |
| 2. DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0xff30579 in /usr/bin/clickhouse | |
| 3. DB::StorageReplicatedMergeTree::write(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>) @ 0xff3068a in /usr/bin/clickhouse | |
| 4. DB::PushingToViewsBlockOutputStream::PushingToViewsBlockOutputStream(std::__1::shared_ptr<DB::IStorage> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::IAST> const&, bool) @ 0xf8b85da in /usr/bin/clickhouse | |
| 5. DB::InterpreterInsertQuery::execute() @ 0xf8b2e2b in /usr/bin/clickhouse | |
| 6. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0x100a5ef5 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0x100a2aa9 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0x100a1cd4 in /usr/bin/clickhouse | |
| 9. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf8b9472 in /usr/bin/clickhouse | |
| 10. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf8c39ab in /usr/bin/clickhouse | |
| 11. DB::SquashingBlockOutputStream::finalize() @ 0xf8c402c in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf8c40c9 in /usr/bin/clickhouse | |
| 13. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xf332d1d in /usr/bin/clickhouse | |
| 14. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf8b6279 in /usr/bin/clickhouse | |
| 15. DB::IBlockInputStream::read() @ 0xf30f452 in /usr/bin/clickhouse | |
| 16. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xf332f76 in /usr/bin/clickhouse | |
| 17. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xfc0eaa5 in /usr/bin/clickhouse | |
| 18. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0x103c36bf in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x103c6e49 in /usr/bin/clickhouse | |
| 20. DB::HTTPServerConnection::run() @ 0x1044acd0 in /usr/bin/clickhouse | |
| 21. Poco::Net::TCPServerConnection::start() @ 0x12a3fd4f in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerDispatcher::run() @ 0x12a417da in /usr/bin/clickhouse | |
| 23. Poco::PooledThread::run() @ 0x12b7ab39 in /usr/bin/clickhouse | |
| 24. Poco::ThreadImpl::runnableEntry(void*) @ 0x12b76b2a in /usr/bin/clickhouse | |
| 25. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 26. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| 2021.06.24 08:39:26.576843 [ 49884 ] {70f3496c-924b-470a-9613-2ee00fca0e69} <Error> DynamicQueryHandler: Code: 242, e.displayText() = DB::Exception: Table is in readonly mode (zookeeper path: /clickhouse/analytics/tables/01/events_local), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8b6cbba in /usr/bin/clickhouse | |
| 1. DB::Exception::Exception<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x8c0b8a3 in /usr/bin/clickhouse | |
| 2. DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0xff30579 in /usr/bin/clickhouse | |
| 3. DB::StorageReplicatedMergeTree::write(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>) @ 0xff3068a in /usr/bin/clickhouse | |
| 4. DB::PushingToViewsBlockOutputStream::PushingToViewsBlockOutputStream(std::__1::shared_ptr<DB::IStorage> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::IAST> const&, bool) @ 0xf8b85da in /usr/bin/clickhouse | |
| 5. DB::InterpreterInsertQuery::execute() @ 0xf8b2e2b in /usr/bin/clickhouse | |
| 6. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0x100a5ef5 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0x100a2aa9 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0x100a1cd4 in /usr/bin/clickhouse | |
| 9. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf8b9472 in /usr/bin/clickhouse | |
| 10. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf8c39ab in /usr/bin/clickhouse | |
| 11. DB::SquashingBlockOutputStream::finalize() @ 0xf8c402c in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf8c40c9 in /usr/bin/clickhouse | |
| 13. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xf332d1d in /usr/bin/clickhouse | |
| 14. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf8b6279 in /usr/bin/clickhouse | |
| 15. DB::IBlockInputStream::read() @ 0xf30f452 in /usr/bin/clickhouse | |
| 16. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xf332f76 in /usr/bin/clickhouse | |
| 17. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xfc0eaa5 in /usr/bin/clickhouse | |
| 18. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0x103c36bf in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x103c6e49 in /usr/bin/clickhouse | |
| 20. DB::HTTPServerConnection::run() @ 0x1044acd0 in /usr/bin/clickhouse | |
| 21. Poco::Net::TCPServerConnection::start() @ 0x12a3fd4f in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerDispatcher::run() @ 0x12a417da in /usr/bin/clickhouse | |
| 23. Poco::PooledThread::run() @ 0x12b7ab39 in /usr/bin/clickhouse | |
| 24. Poco::ThreadImpl::runnableEntry(void*) @ 0x12b76b2a in /usr/bin/clickhouse | |
| 25. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 26. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.6.5.37 (official build)) | |
| 2021.06.24 08:39:26.582029 [ 49884 ] {5fe1c1a7-7938-47a5-b276-6928e30e2c04} <Warning> ClusterProxy::SelectStreamFactory: Local replica of shard 1 is stale (delay: 1624523966s.) | |
| 2021.06.24 08:39:27.671920 [ 49884 ] {6e27eea8-2697-4fc2-b1ec-ac39393783d6} <Error> executeQuery: Code: 242, e.displayText() = DB::Exception: Table is in readonly mode (zookeeper path: /clickhouse/analytics/tables/01/events_local) (version 21.6.5.37 (official build)) (from 172.31.18.125:35352) (in query: INSERT INTO events (`id`, `event_type`, `date`, `customer_id`, `engine_id`, `query`, `display_width_px`, `display_height_px`, `user_agent`, `referrer_url`, `referrer_title`, `browser_language`, `ip_address`, `page_number`, `host_name`, `session_id`, `query_session_id`, `query_raw`, `device_type`, `geo_continent_name`, `geo_city_name`, `geo_country_iso_code`, `geo_region_name`, `geo_latitude`, `geo_longitude`, `click_link_url`, `click_link_index`, `click_clicked_object_id`, `click_title`, `click_query_id`, `click_context`, `query_type`, `query_total_banners`, `query_total_documents`, `query_response_time_ms`, `query_actual_query`, `query_quick_link_id`, `query_results_per_page`, `query_environment`, `query_request_type`, `query_banners`, `query_billable`, `application_name`, `application_version`, `http_status_code`, `aws_region`) FORMAT RowBinary ), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8b6cbba in /usr/bin/clickhouse | |
| 1. DB::Exception::Exception<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x8c0b8a3 in /usr/bin/clickhouse | |
| 2. DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0xff30579 in /usr/bin/clickhouse | |
| 3. DB::StorageReplicatedMergeTree::write(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>) @ 0xff3068a in /usr/bin/clickhouse | |
| 4. DB::PushingToViewsBlockOutputStream::PushingToViewsBlockOutputStream(std::__1::shared_ptr<DB::IStorage> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::IAST> const&, bool) @ 0xf8b85da in /usr/bin/clickhouse | |
| 5. DB::InterpreterInsertQuery::execute() @ 0xf8b2e2b in /usr/bin/clickhouse | |
| 6. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0x100a5ef5 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0x100a2aa9 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0x100a1cd4 in /usr/bin/clickhouse | |
| 9. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf8b9472 in /usr/bin/clickhouse | |
| 10. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf8c39ab in /usr/bin/clickhouse | |
| 11. DB::SquashingBlockOutputStream::finalize() @ 0xf8c402c in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf8c40c9 in /usr/bin/clickhouse | |
| 13. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xf332d1d in /usr/bin/clickhouse | |
| 14. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf8b6279 in /usr/bin/clickhouse | |
| 15. DB::IBlockInputStream::read() @ 0xf30f452 in /usr/bin/clickhouse | |
| 16. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xf332f76 in /usr/bin/clickhouse | |
| 17. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xfc0eaa5 in /usr/bin/clickhouse | |
| 18. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0x103c36bf in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x103c6e49 in /usr/bin/clickhouse | |
| 20. DB::HTTPServerConnection::run() @ 0x1044acd0 in /usr/bin/clickhouse | |
| 21. Poco::Net::TCPServerConnection::start() @ 0x12a3fd4f in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerDispatcher::run() @ 0x12a417da in /usr/bin/clickhouse | |
| 23. Poco::PooledThread::run() @ 0x12b7ab39 in /usr/bin/clickhouse | |
| 24. Poco::ThreadImpl::runnableEntry(void*) @ 0x12b76b2a in /usr/bin/clickhouse | |
| 25. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 26. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| 2021.06.24 08:39:27.672105 [ 49884 ] {6e27eea8-2697-4fc2-b1ec-ac39393783d6} <Error> DynamicQueryHandler: Code: 242, e.displayText() = DB::Exception: Table is in readonly mode (zookeeper path: /clickhouse/analytics/tables/01/events_local), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8b6cbba in /usr/bin/clickhouse | |
| 1. DB::Exception::Exception<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x8c0b8a3 in /usr/bin/clickhouse | |
| 2. DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0xff30579 in /usr/bin/clickhouse | |
| 3. DB::StorageReplicatedMergeTree::write(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>) @ 0xff3068a in /usr/bin/clickhouse | |
| 4. DB::PushingToViewsBlockOutputStream::PushingToViewsBlockOutputStream(std::__1::shared_ptr<DB::IStorage> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::IAST> const&, bool) @ 0xf8b85da in /usr/bin/clickhouse | |
| 5. DB::InterpreterInsertQuery::execute() @ 0xf8b2e2b in /usr/bin/clickhouse | |
| 6. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0x100a5ef5 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0x100a2aa9 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0x100a1cd4 in /usr/bin/clickhouse | |
| 9. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf8b9472 in /usr/bin/clickhouse | |
| 10. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf8c39ab in /usr/bin/clickhouse | |
| 11. DB::SquashingBlockOutputStream::finalize() @ 0xf8c402c in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf8c40c9 in /usr/bin/clickhouse | |
| 13. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xf332d1d in /usr/bin/clickhouse | |
| 14. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf8b6279 in /usr/bin/clickhouse | |
| 15. DB::IBlockInputStream::read() @ 0xf30f452 in /usr/bin/clickhouse | |
| 16. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xf332f76 in /usr/bin/clickhouse | |
| 17. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xfc0eaa5 in /usr/bin/clickhouse | |
| 18. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0x103c36bf in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x103c6e49 in /usr/bin/clickhouse | |
| 20. DB::HTTPServerConnection::run() @ 0x1044acd0 in /usr/bin/clickhouse | |
| 21. Poco::Net::TCPServerConnection::start() @ 0x12a3fd4f in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerDispatcher::run() @ 0x12a417da in /usr/bin/clickhouse | |
| 23. Poco::PooledThread::run() @ 0x12b7ab39 in /usr/bin/clickhouse | |
| 24. Poco::ThreadImpl::runnableEntry(void*) @ 0x12b76b2a in /usr/bin/clickhouse | |
| 25. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 26. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.6.5.37 (official build)) | |
| 2021.06.24 08:39:28.221187 [ 49889 ] {} <Error> analytics.events_local (ReplicatedMergeTreeRestartingThread): Couldn't start replication (table will be in readonly mode): Replica /clickhouse/analytics/tables/01/events_local/replicas/01 appears to be already active (host: ip-172-31-23-246.eu-central-1.compute.internal, port: 9009, tcp_port: 9000, database: analytics, table: events_local, scheme: http, ). If you're sure it's not, try again in a minute or remove znode /clickhouse/analytics/tables/01/events_local/replicas/01/is_active manually. Code: 224, e.displayText() = DB::Exception: Replica /clickhouse/analytics/tables/01/events_local/replicas/01 appears to be already active (host: ip-172-31-23-246.eu-central-1.compute.internal, port: 9009, tcp_port: 9000, database: analytics, table: events_local, scheme: http, ). If you're sure it's not, try again in a minute or remove znode /clickhouse/analytics/tables/01/events_local/replicas/01/is_active manually, Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8b6cbba in /usr/bin/clickhouse | |
| 1. DB::Exception::Exception<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&>(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&) @ 0x8bf3513 in /usr/bin/clickhouse | |
| 2. DB::ReplicatedMergeTreeRestartingThread::activateReplica() @ 0x10308451 in /usr/bin/clickhouse | |
| 3. DB::ReplicatedMergeTreeRestartingThread::tryStartup() @ 0x10306301 in /usr/bin/clickhouse | |
| 4. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x103051c5 in /usr/bin/clickhouse | |
| 5. DB::BackgroundSchedulePoolTaskInfo::execute() @ 0xf5596a0 in /usr/bin/clickhouse | |
| 6. DB::BackgroundSchedulePool::threadFunction() @ 0xf55b6f7 in /usr/bin/clickhouse | |
| 7. ? @ 0xf55c474 in /usr/bin/clickhouse | |
| 8. ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0x8bacedf in /usr/bin/clickhouse | |
| 9. ? @ 0x8bb0403 in /usr/bin/clickhouse | |
| 10. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 11. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.6.5.37 (official build)) | |
| 2021.06.24 08:39:28.714839 [ 49884 ] {17c999f1-2c5d-4e95-99c9-51e1ae4f75e8} <Error> executeQuery: Code: 242, e.displayText() = DB::Exception: Table is in readonly mode (zookeeper path: /clickhouse/analytics/tables/01/events_local) (version 21.6.5.37 (official build)) (from 172.31.18.125:35360) (in query: INSERT INTO events (`id`, `event_type`, `date`, `customer_id`, `engine_id`, `query`, `display_width_px`, `display_height_px`, `user_agent`, `referrer_url`, `referrer_title`, `browser_language`, `ip_address`, `page_number`, `host_name`, `session_id`, `query_session_id`, `query_raw`, `device_type`, `geo_continent_name`, `geo_city_name`, `geo_country_iso_code`, `geo_region_name`, `geo_latitude`, `geo_longitude`, `click_link_url`, `click_link_index`, `click_clicked_object_id`, `click_title`, `click_query_id`, `click_context`, `query_type`, `query_total_banners`, `query_total_documents`, `query_response_time_ms`, `query_actual_query`, `query_quick_link_id`, `query_results_per_page`, `query_environment`, `query_request_type`, `query_banners`, `query_billable`, `application_name`, `application_version`, `http_status_code`, `aws_region`) FORMAT RowBinary ), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8b6cbba in /usr/bin/clickhouse | |
| 1. DB::Exception::Exception<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x8c0b8a3 in /usr/bin/clickhouse | |
| 2. DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0xff30579 in /usr/bin/clickhouse | |
| 3. DB::StorageReplicatedMergeTree::write(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>) @ 0xff3068a in /usr/bin/clickhouse | |
| 4. DB::PushingToViewsBlockOutputStream::PushingToViewsBlockOutputStream(std::__1::shared_ptr<DB::IStorage> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::IAST> const&, bool) @ 0xf8b85da in /usr/bin/clickhouse | |
| 5. DB::InterpreterInsertQuery::execute() @ 0xf8b2e2b in /usr/bin/clickhouse | |
| 6. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0x100a5ef5 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0x100a2aa9 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0x100a1cd4 in /usr/bin/clickhouse | |
| 9. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf8b9472 in /usr/bin/clickhouse | |
| 10. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf8c39ab in /usr/bin/clickhouse | |
| 11. DB::SquashingBlockOutputStream::finalize() @ 0xf8c402c in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf8c40c9 in /usr/bin/clickhouse | |
| 13. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xf332d1d in /usr/bin/clickhouse | |
| 14. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf8b6279 in /usr/bin/clickhouse | |
| 15. DB::IBlockInputStream::read() @ 0xf30f452 in /usr/bin/clickhouse | |
| 16. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xf332f76 in /usr/bin/clickhouse | |
| 17. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xfc0eaa5 in /usr/bin/clickhouse | |
| 18. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0x103c36bf in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x103c6e49 in /usr/bin/clickhouse | |
| 20. DB::HTTPServerConnection::run() @ 0x1044acd0 in /usr/bin/clickhouse | |
| 21. Poco::Net::TCPServerConnection::start() @ 0x12a3fd4f in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerDispatcher::run() @ 0x12a417da in /usr/bin/clickhouse | |
| 23. Poco::PooledThread::run() @ 0x12b7ab39 in /usr/bin/clickhouse | |
| 24. Poco::ThreadImpl::runnableEntry(void*) @ 0x12b76b2a in /usr/bin/clickhouse | |
| 25. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 26. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| 2021.06.24 08:39:28.715018 [ 49884 ] {17c999f1-2c5d-4e95-99c9-51e1ae4f75e8} <Error> DynamicQueryHandler: Code: 242, e.displayText() = DB::Exception: Table is in readonly mode (zookeeper path: /clickhouse/analytics/tables/01/events_local), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8b6cbba in /usr/bin/clickhouse | |
| 1. DB::Exception::Exception<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x8c0b8a3 in /usr/bin/clickhouse | |
| 2. DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0xff30579 in /usr/bin/clickhouse | |
| 3. DB::StorageReplicatedMergeTree::write(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>) @ 0xff3068a in /usr/bin/clickhouse | |
| 4. DB::PushingToViewsBlockOutputStream::PushingToViewsBlockOutputStream(std::__1::shared_ptr<DB::IStorage> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::IAST> const&, bool) @ 0xf8b85da in /usr/bin/clickhouse | |
| 5. DB::InterpreterInsertQuery::execute() @ 0xf8b2e2b in /usr/bin/clickhouse | |
| 6. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0x100a5ef5 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0x100a2aa9 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0x100a1cd4 in /usr/bin/clickhouse | |
| 9. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf8b9472 in /usr/bin/clickhouse | |
| 10. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf8c39ab in /usr/bin/clickhouse | |
| 11. DB::SquashingBlockOutputStream::finalize() @ 0xf8c402c in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf8c40c9 in /usr/bin/clickhouse | |
| 13. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xf332d1d in /usr/bin/clickhouse | |
| 14. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf8b6279 in /usr/bin/clickhouse | |
| 15. DB::IBlockInputStream::read() @ 0xf30f452 in /usr/bin/clickhouse | |
| 16. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xf332f76 in /usr/bin/clickhouse | |
| 17. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xfc0eaa5 in /usr/bin/clickhouse | |
| 18. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0x103c36bf in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x103c6e49 in /usr/bin/clickhouse | |
| 20. DB::HTTPServerConnection::run() @ 0x1044acd0 in /usr/bin/clickhouse | |
| 21. Poco::Net::TCPServerConnection::start() @ 0x12a3fd4f in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerDispatcher::run() @ 0x12a417da in /usr/bin/clickhouse | |
| 23. Poco::PooledThread::run() @ 0x12b7ab39 in /usr/bin/clickhouse | |
| 24. Poco::ThreadImpl::runnableEntry(void*) @ 0x12b76b2a in /usr/bin/clickhouse | |
| 25. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 26. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.6.5.37 (official build)) | |
| 2021.06.24 08:39:28.978738 [ 49884 ] {a3cf571a-9aa5-4e87-a016-eee882d56cd3} <Warning> ClusterProxy::SelectStreamFactory: Local replica of shard 1 is stale (delay: 1624523968s.) | |
| 2021.06.24 08:39:29.748322 [ 49884 ] {e3c57e45-df03-4d72-9c9b-32f85982162c} <Error> executeQuery: Code: 242, e.displayText() = DB::Exception: Table is in readonly mode (zookeeper path: /clickhouse/analytics/tables/01/events_local) (version 21.6.5.37 (official build)) (from 172.31.18.125:35372) (in query: INSERT INTO events (`id`, `event_type`, `date`, `customer_id`, `engine_id`, `query`, `display_width_px`, `display_height_px`, `user_agent`, `referrer_url`, `referrer_title`, `browser_language`, `ip_address`, `page_number`, `host_name`, `session_id`, `query_session_id`, `query_raw`, `device_type`, `geo_continent_name`, `geo_city_name`, `geo_country_iso_code`, `geo_region_name`, `geo_latitude`, `geo_longitude`, `click_link_url`, `click_link_index`, `click_clicked_object_id`, `click_title`, `click_query_id`, `click_context`, `query_type`, `query_total_banners`, `query_total_documents`, `query_response_time_ms`, `query_actual_query`, `query_quick_link_id`, `query_results_per_page`, `query_environment`, `query_request_type`, `query_banners`, `query_billable`, `application_name`, `application_version`, `http_status_code`, `aws_region`) FORMAT RowBinary ), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8b6cbba in /usr/bin/clickhouse | |
| 1. DB::Exception::Exception<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x8c0b8a3 in /usr/bin/clickhouse | |
| 2. DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0xff30579 in /usr/bin/clickhouse | |
| 3. DB::StorageReplicatedMergeTree::write(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>) @ 0xff3068a in /usr/bin/clickhouse | |
| 4. DB::PushingToViewsBlockOutputStream::PushingToViewsBlockOutputStream(std::__1::shared_ptr<DB::IStorage> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::IAST> const&, bool) @ 0xf8b85da in /usr/bin/clickhouse | |
| 5. DB::InterpreterInsertQuery::execute() @ 0xf8b2e2b in /usr/bin/clickhouse | |
| 6. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0x100a5ef5 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0x100a2aa9 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0x100a1cd4 in /usr/bin/clickhouse | |
| 9. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf8b9472 in /usr/bin/clickhouse | |
| 10. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf8c39ab in /usr/bin/clickhouse | |
| 11. DB::SquashingBlockOutputStream::finalize() @ 0xf8c402c in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf8c40c9 in /usr/bin/clickhouse | |
| 13. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xf332d1d in /usr/bin/clickhouse | |
| 14. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf8b6279 in /usr/bin/clickhouse | |
| 15. DB::IBlockInputStream::read() @ 0xf30f452 in /usr/bin/clickhouse | |
| 16. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xf332f76 in /usr/bin/clickhouse | |
| 17. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xfc0eaa5 in /usr/bin/clickhouse | |
| 18. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0x103c36bf in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x103c6e49 in /usr/bin/clickhouse | |
| 20. DB::HTTPServerConnection::run() @ 0x1044acd0 in /usr/bin/clickhouse | |
| 21. Poco::Net::TCPServerConnection::start() @ 0x12a3fd4f in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerDispatcher::run() @ 0x12a417da in /usr/bin/clickhouse | |
| 23. Poco::PooledThread::run() @ 0x12b7ab39 in /usr/bin/clickhouse | |
| 24. Poco::ThreadImpl::runnableEntry(void*) @ 0x12b76b2a in /usr/bin/clickhouse | |
| 25. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 26. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| 2021.06.24 08:39:29.748508 [ 49884 ] {e3c57e45-df03-4d72-9c9b-32f85982162c} <Error> DynamicQueryHandler: Code: 242, e.displayText() = DB::Exception: Table is in readonly mode (zookeeper path: /clickhouse/analytics/tables/01/events_local), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8b6cbba in /usr/bin/clickhouse | |
| 1. DB::Exception::Exception<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x8c0b8a3 in /usr/bin/clickhouse | |
| 2. DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0xff30579 in /usr/bin/clickhouse | |
| 3. DB::StorageReplicatedMergeTree::write(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>) @ 0xff3068a in /usr/bin/clickhouse | |
| 4. DB::PushingToViewsBlockOutputStream::PushingToViewsBlockOutputStream(std::__1::shared_ptr<DB::IStorage> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::IAST> const&, bool) @ 0xf8b85da in /usr/bin/clickhouse | |
| 5. DB::InterpreterInsertQuery::execute() @ 0xf8b2e2b in /usr/bin/clickhouse | |
| 6. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0x100a5ef5 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0x100a2aa9 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0x100a1cd4 in /usr/bin/clickhouse | |
| 9. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf8b9472 in /usr/bin/clickhouse | |
| 10. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf8c39ab in /usr/bin/clickhouse | |
| 11. DB::SquashingBlockOutputStream::finalize() @ 0xf8c402c in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf8c40c9 in /usr/bin/clickhouse | |
| 13. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xf332d1d in /usr/bin/clickhouse | |
| 14. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf8b6279 in /usr/bin/clickhouse | |
| 15. DB::IBlockInputStream::read() @ 0xf30f452 in /usr/bin/clickhouse | |
| 16. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xf332f76 in /usr/bin/clickhouse | |
| 17. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xfc0eaa5 in /usr/bin/clickhouse | |
| 18. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0x103c36bf in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x103c6e49 in /usr/bin/clickhouse | |
| 20. DB::HTTPServerConnection::run() @ 0x1044acd0 in /usr/bin/clickhouse | |
| 21. Poco::Net::TCPServerConnection::start() @ 0x12a3fd4f in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerDispatcher::run() @ 0x12a417da in /usr/bin/clickhouse | |
| 23. Poco::PooledThread::run() @ 0x12b7ab39 in /usr/bin/clickhouse | |
| 24. Poco::ThreadImpl::runnableEntry(void*) @ 0x12b76b2a in /usr/bin/clickhouse | |
| 25. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 26. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.6.5.37 (official build)) | |
| 2021.06.24 08:39:29.765010 [ 49884 ] {5c49f4a5-30b9-4049-a994-73ac84f26bfe} <Warning> ClusterProxy::SelectStreamFactory: Local replica of shard 1 is stale (delay: 1624523969s.) | |
| 2021.06.24 08:39:30.019488 [ 49884 ] {86cdbcf7-f781-45f2-ab52-0c4b7422f764} <Error> executeQuery: Code: 242, e.displayText() = DB::Exception: Table is in readonly mode (zookeeper path: /clickhouse/analytics/tables/01/events_local) (version 21.6.5.37 (official build)) (from 172.31.18.125:35384) (in query: INSERT INTO events (`id`, `event_type`, `date`, `customer_id`, `engine_id`, `query`, `display_width_px`, `display_height_px`, `user_agent`, `referrer_url`, `referrer_title`, `browser_language`, `ip_address`, `page_number`, `host_name`, `session_id`, `query_session_id`, `query_raw`, `device_type`, `geo_continent_name`, `geo_city_name`, `geo_country_iso_code`, `geo_region_name`, `geo_latitude`, `geo_longitude`, `click_link_url`, `click_link_index`, `click_clicked_object_id`, `click_title`, `click_query_id`, `click_context`, `query_type`, `query_total_banners`, `query_total_documents`, `query_response_time_ms`, `query_actual_query`, `query_quick_link_id`, `query_results_per_page`, `query_environment`, `query_request_type`, `query_banners`, `query_billable`, `application_name`, `application_version`, `http_status_code`, `aws_region`) FORMAT RowBinary ), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8b6cbba in /usr/bin/clickhouse | |
| 1. DB::Exception::Exception<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x8c0b8a3 in /usr/bin/clickhouse | |
| 2. DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0xff30579 in /usr/bin/clickhouse | |
| 3. DB::StorageReplicatedMergeTree::write(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>) @ 0xff3068a in /usr/bin/clickhouse | |
| 4. DB::PushingToViewsBlockOutputStream::PushingToViewsBlockOutputStream(std::__1::shared_ptr<DB::IStorage> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::IAST> const&, bool) @ 0xf8b85da in /usr/bin/clickhouse | |
| 5. DB::InterpreterInsertQuery::execute() @ 0xf8b2e2b in /usr/bin/clickhouse | |
| 6. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0x100a5ef5 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0x100a2aa9 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0x100a1cd4 in /usr/bin/clickhouse | |
| 9. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf8b9472 in /usr/bin/clickhouse | |
| 10. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf8c39ab in /usr/bin/clickhouse | |
| 11. DB::SquashingBlockOutputStream::finalize() @ 0xf8c402c in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf8c40c9 in /usr/bin/clickhouse | |
| 13. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xf332d1d in /usr/bin/clickhouse | |
| 14. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf8b6279 in /usr/bin/clickhouse | |
| 15. DB::IBlockInputStream::read() @ 0xf30f452 in /usr/bin/clickhouse | |
| 16. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xf332f76 in /usr/bin/clickhouse | |
| 17. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xfc0eaa5 in /usr/bin/clickhouse | |
| 18. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0x103c36bf in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x103c6e49 in /usr/bin/clickhouse | |
| 20. DB::HTTPServerConnection::run() @ 0x1044acd0 in /usr/bin/clickhouse | |
| 21. Poco::Net::TCPServerConnection::start() @ 0x12a3fd4f in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerDispatcher::run() @ 0x12a417da in /usr/bin/clickhouse | |
| 23. Poco::PooledThread::run() @ 0x12b7ab39 in /usr/bin/clickhouse | |
| 24. Poco::ThreadImpl::runnableEntry(void*) @ 0x12b76b2a in /usr/bin/clickhouse | |
| 25. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 26. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| 2021.06.24 08:39:30.019679 [ 49884 ] {86cdbcf7-f781-45f2-ab52-0c4b7422f764} <Error> DynamicQueryHandler: Code: 242, e.displayText() = DB::Exception: Table is in readonly mode (zookeeper path: /clickhouse/analytics/tables/01/events_local), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8b6cbba in /usr/bin/clickhouse | |
| 1. DB::Exception::Exception<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x8c0b8a3 in /usr/bin/clickhouse | |
| 2. DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0xff30579 in /usr/bin/clickhouse | |
| 3. DB::StorageReplicatedMergeTree::write(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>) @ 0xff3068a in /usr/bin/clickhouse | |
| 4. DB::PushingToViewsBlockOutputStream::PushingToViewsBlockOutputStream(std::__1::shared_ptr<DB::IStorage> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::IAST> const&, bool) @ 0xf8b85da in /usr/bin/clickhouse | |
| 5. DB::InterpreterInsertQuery::execute() @ 0xf8b2e2b in /usr/bin/clickhouse | |
| 6. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0x100a5ef5 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0x100a2aa9 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0x100a1cd4 in /usr/bin/clickhouse | |
| 9. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf8b9472 in /usr/bin/clickhouse | |
| 10. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf8c39ab in /usr/bin/clickhouse | |
| 11. DB::SquashingBlockOutputStream::finalize() @ 0xf8c402c in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf8c40c9 in /usr/bin/clickhouse | |
| 13. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xf332d1d in /usr/bin/clickhouse | |
| 14. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf8b6279 in /usr/bin/clickhouse | |
| 15. DB::IBlockInputStream::read() @ 0xf30f452 in /usr/bin/clickhouse | |
| 16. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xf332f76 in /usr/bin/clickhouse | |
| 17. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xfc0eaa5 in /usr/bin/clickhouse | |
| 18. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0x103c36bf in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x103c6e49 in /usr/bin/clickhouse | |
| 20. DB::HTTPServerConnection::run() @ 0x1044acd0 in /usr/bin/clickhouse | |
| 21. Poco::Net::TCPServerConnection::start() @ 0x12a3fd4f in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerDispatcher::run() @ 0x12a417da in /usr/bin/clickhouse | |
| 23. Poco::PooledThread::run() @ 0x12b7ab39 in /usr/bin/clickhouse | |
| 24. Poco::ThreadImpl::runnableEntry(void*) @ 0x12b76b2a in /usr/bin/clickhouse | |
| 25. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 26. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.6.5.37 (official build)) | |
| 2021.06.24 08:39:30.029400 [ 49884 ] {45dc90cf-50af-4b08-adc5-67ec4bf962b4} <Warning> ClusterProxy::SelectStreamFactory: Local replica of shard 1 is stale (delay: 1624523970s.) | |
| 2021.06.24 08:39:30.814170 [ 49884 ] {64a670a3-736b-4f94-8963-758e4dc2cc94} <Error> executeQuery: Code: 242, e.displayText() = DB::Exception: Table is in readonly mode (zookeeper path: /clickhouse/analytics/tables/01/events_local) (version 21.6.5.37 (official build)) (from 172.31.18.125:35396) (in query: INSERT INTO events (`id`, `event_type`, `date`, `customer_id`, `engine_id`, `query`, `display_width_px`, `display_height_px`, `user_agent`, `referrer_url`, `referrer_title`, `browser_language`, `ip_address`, `page_number`, `host_name`, `session_id`, `query_session_id`, `query_raw`, `device_type`, `geo_continent_name`, `geo_city_name`, `geo_country_iso_code`, `geo_region_name`, `geo_latitude`, `geo_longitude`, `click_link_url`, `click_link_index`, `click_clicked_object_id`, `click_title`, `click_query_id`, `click_context`, `query_type`, `query_total_banners`, `query_total_documents`, `query_response_time_ms`, `query_actual_query`, `query_quick_link_id`, `query_results_per_page`, `query_environment`, `query_request_type`, `query_banners`, `query_billable`, `application_name`, `application_version`, `http_status_code`, `aws_region`) FORMAT RowBinary ), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8b6cbba in /usr/bin/clickhouse | |
| 1. DB::Exception::Exception<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x8c0b8a3 in /usr/bin/clickhouse | |
| 2. DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0xff30579 in /usr/bin/clickhouse | |
| 3. DB::StorageReplicatedMergeTree::write(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>) @ 0xff3068a in /usr/bin/clickhouse | |
| 4. DB::PushingToViewsBlockOutputStream::PushingToViewsBlockOutputStream(std::__1::shared_ptr<DB::IStorage> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::IAST> const&, bool) @ 0xf8b85da in /usr/bin/clickhouse | |
| 5. DB::InterpreterInsertQuery::execute() @ 0xf8b2e2b in /usr/bin/clickhouse | |
| 6. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0x100a5ef5 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0x100a2aa9 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0x100a1cd4 in /usr/bin/clickhouse | |
| 9. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf8b9472 in /usr/bin/clickhouse | |
| 10. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf8c39ab in /usr/bin/clickhouse | |
| 11. DB::SquashingBlockOutputStream::finalize() @ 0xf8c402c in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf8c40c9 in /usr/bin/clickhouse | |
| 13. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xf332d1d in /usr/bin/clickhouse | |
| 14. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf8b6279 in /usr/bin/clickhouse | |
| 15. DB::IBlockInputStream::read() @ 0xf30f452 in /usr/bin/clickhouse | |
| 16. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xf332f76 in /usr/bin/clickhouse | |
| 17. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xfc0eaa5 in /usr/bin/clickhouse | |
| 18. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0x103c36bf in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x103c6e49 in /usr/bin/clickhouse | |
| 20. DB::HTTPServerConnection::run() @ 0x1044acd0 in /usr/bin/clickhouse | |
| 21. Poco::Net::TCPServerConnection::start() @ 0x12a3fd4f in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerDispatcher::run() @ 0x12a417da in /usr/bin/clickhouse | |
| 23. Poco::PooledThread::run() @ 0x12b7ab39 in /usr/bin/clickhouse | |
| 24. Poco::ThreadImpl::runnableEntry(void*) @ 0x12b76b2a in /usr/bin/clickhouse | |
| 25. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 26. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| 2021.06.24 08:39:30.814351 [ 49884 ] {64a670a3-736b-4f94-8963-758e4dc2cc94} <Error> DynamicQueryHandler: Code: 242, e.displayText() = DB::Exception: Table is in readonly mode (zookeeper path: /clickhouse/analytics/tables/01/events_local), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8b6cbba in /usr/bin/clickhouse | |
| 1. DB::Exception::Exception<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x8c0b8a3 in /usr/bin/clickhouse | |
| 2. DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0xff30579 in /usr/bin/clickhouse | |
| 3. DB::StorageReplicatedMergeTree::write(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>) @ 0xff3068a in /usr/bin/clickhouse | |
| 4. DB::PushingToViewsBlockOutputStream::PushingToViewsBlockOutputStream(std::__1::shared_ptr<DB::IStorage> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::IAST> const&, bool) @ 0xf8b85da in /usr/bin/clickhouse | |
| 5. DB::InterpreterInsertQuery::execute() @ 0xf8b2e2b in /usr/bin/clickhouse | |
| 6. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0x100a5ef5 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0x100a2aa9 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0x100a1cd4 in /usr/bin/clickhouse | |
| 9. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf8b9472 in /usr/bin/clickhouse | |
| 10. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf8c39ab in /usr/bin/clickhouse | |
| 11. DB::SquashingBlockOutputStream::finalize() @ 0xf8c402c in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf8c40c9 in /usr/bin/clickhouse | |
| 13. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xf332d1d in /usr/bin/clickhouse | |
| 14. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf8b6279 in /usr/bin/clickhouse | |
| 15. DB::IBlockInputStream::read() @ 0xf30f452 in /usr/bin/clickhouse | |
| 16. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xf332f76 in /usr/bin/clickhouse | |
| 17. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xfc0eaa5 in /usr/bin/clickhouse | |
| 18. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0x103c36bf in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x103c6e49 in /usr/bin/clickhouse | |
| 20. DB::HTTPServerConnection::run() @ 0x1044acd0 in /usr/bin/clickhouse | |
| 21. Poco::Net::TCPServerConnection::start() @ 0x12a3fd4f in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerDispatcher::run() @ 0x12a417da in /usr/bin/clickhouse | |
| 23. Poco::PooledThread::run() @ 0x12b7ab39 in /usr/bin/clickhouse | |
| 24. Poco::ThreadImpl::runnableEntry(void*) @ 0x12b76b2a in /usr/bin/clickhouse | |
| 25. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 26. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.6.5.37 (official build)) | |
| 2021.06.24 08:39:31.073179 [ 49884 ] {a9c9105a-8a97-443d-91f7-2f9847dd975c} <Error> executeQuery: Code: 242, e.displayText() = DB::Exception: Table is in readonly mode (zookeeper path: /clickhouse/analytics/tables/01/events_local) (version 21.6.5.37 (official build)) (from 172.31.18.125:35404) (in query: INSERT INTO events (`id`, `event_type`, `date`, `customer_id`, `engine_id`, `query`, `display_width_px`, `display_height_px`, `user_agent`, `referrer_url`, `referrer_title`, `browser_language`, `ip_address`, `page_number`, `host_name`, `session_id`, `query_session_id`, `query_raw`, `device_type`, `geo_continent_name`, `geo_city_name`, `geo_country_iso_code`, `geo_region_name`, `geo_latitude`, `geo_longitude`, `click_link_url`, `click_link_index`, `click_clicked_object_id`, `click_title`, `click_query_id`, `click_context`, `query_type`, `query_total_banners`, `query_total_documents`, `query_response_time_ms`, `query_actual_query`, `query_quick_link_id`, `query_results_per_page`, `query_environment`, `query_request_type`, `query_banners`, `query_billable`, `application_name`, `application_version`, `http_status_code`, `aws_region`) FORMAT RowBinary ), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8b6cbba in /usr/bin/clickhouse | |
| 1. DB::Exception::Exception<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x8c0b8a3 in /usr/bin/clickhouse | |
| 2. DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0xff30579 in /usr/bin/clickhouse | |
| 3. DB::StorageReplicatedMergeTree::write(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>) @ 0xff3068a in /usr/bin/clickhouse | |
| 4. DB::PushingToViewsBlockOutputStream::PushingToViewsBlockOutputStream(std::__1::shared_ptr<DB::IStorage> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::IAST> const&, bool) @ 0xf8b85da in /usr/bin/clickhouse | |
| 5. DB::InterpreterInsertQuery::execute() @ 0xf8b2e2b in /usr/bin/clickhouse | |
| 6. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0x100a5ef5 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0x100a2aa9 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0x100a1cd4 in /usr/bin/clickhouse | |
| 9. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf8b9472 in /usr/bin/clickhouse | |
| 10. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf8c39ab in /usr/bin/clickhouse | |
| 11. DB::SquashingBlockOutputStream::finalize() @ 0xf8c402c in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf8c40c9 in /usr/bin/clickhouse | |
| 13. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xf332d1d in /usr/bin/clickhouse | |
| 14. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf8b6279 in /usr/bin/clickhouse | |
| 15. DB::IBlockInputStream::read() @ 0xf30f452 in /usr/bin/clickhouse | |
| 16. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xf332f76 in /usr/bin/clickhouse | |
| 17. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xfc0eaa5 in /usr/bin/clickhouse | |
| 18. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0x103c36bf in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x103c6e49 in /usr/bin/clickhouse | |
| 20. DB::HTTPServerConnection::run() @ 0x1044acd0 in /usr/bin/clickhouse | |
| 21. Poco::Net::TCPServerConnection::start() @ 0x12a3fd4f in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerDispatcher::run() @ 0x12a417da in /usr/bin/clickhouse | |
| 23. Poco::PooledThread::run() @ 0x12b7ab39 in /usr/bin/clickhouse | |
| 24. Poco::ThreadImpl::runnableEntry(void*) @ 0x12b76b2a in /usr/bin/clickhouse | |
| 25. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 26. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| 2021.06.24 08:39:31.073355 [ 49884 ] {a9c9105a-8a97-443d-91f7-2f9847dd975c} <Error> DynamicQueryHandler: Code: 242, e.displayText() = DB::Exception: Table is in readonly mode (zookeeper path: /clickhouse/analytics/tables/01/events_local), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8b6cbba in /usr/bin/clickhouse | |
| 1. DB::Exception::Exception<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x8c0b8a3 in /usr/bin/clickhouse | |
| 2. DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0xff30579 in /usr/bin/clickhouse | |
| 3. DB::StorageReplicatedMergeTree::write(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>) @ 0xff3068a in /usr/bin/clickhouse | |
| 4. DB::PushingToViewsBlockOutputStream::PushingToViewsBlockOutputStream(std::__1::shared_ptr<DB::IStorage> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::IAST> const&, bool) @ 0xf8b85da in /usr/bin/clickhouse | |
| 5. DB::InterpreterInsertQuery::execute() @ 0xf8b2e2b in /usr/bin/clickhouse | |
| 6. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0x100a5ef5 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0x100a2aa9 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0x100a1cd4 in /usr/bin/clickhouse | |
| 9. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf8b9472 in /usr/bin/clickhouse | |
| 10. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf8c39ab in /usr/bin/clickhouse | |
| 11. DB::SquashingBlockOutputStream::finalize() @ 0xf8c402c in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf8c40c9 in /usr/bin/clickhouse | |
| 13. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xf332d1d in /usr/bin/clickhouse | |
| 14. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf8b6279 in /usr/bin/clickhouse | |
| 15. DB::IBlockInputStream::read() @ 0xf30f452 in /usr/bin/clickhouse | |
| 16. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xf332f76 in /usr/bin/clickhouse | |
| 17. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xfc0eaa5 in /usr/bin/clickhouse | |
| 18. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0x103c36bf in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x103c6e49 in /usr/bin/clickhouse | |
| 20. DB::HTTPServerConnection::run() @ 0x1044acd0 in /usr/bin/clickhouse | |
| 21. Poco::Net::TCPServerConnection::start() @ 0x12a3fd4f in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerDispatcher::run() @ 0x12a417da in /usr/bin/clickhouse | |
| 23. Poco::PooledThread::run() @ 0x12b7ab39 in /usr/bin/clickhouse | |
| 24. Poco::ThreadImpl::runnableEntry(void*) @ 0x12b76b2a in /usr/bin/clickhouse | |
| 25. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 26. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.6.5.37 (official build)) | |
| 2021.06.24 08:39:31.821338 [ 49884 ] {75d1e71d-82bb-4c64-ace4-48b86f6f06a3} <Warning> ClusterProxy::SelectStreamFactory: Local replica of shard 1 is stale (delay: 1624523971s.) | |
| 2021.06.24 08:39:32.089877 [ 49884 ] {895582f8-73ac-4fb3-8819-d3afb590f566} <Error> executeQuery: Code: 242, e.displayText() = DB::Exception: Table is in readonly mode (zookeeper path: /clickhouse/analytics/tables/01/events_local) (version 21.6.5.37 (official build)) (from 172.31.18.125:35416) (in query: INSERT INTO events (`id`, `event_type`, `date`, `customer_id`, `engine_id`, `query`, `display_width_px`, `display_height_px`, `user_agent`, `referrer_url`, `referrer_title`, `browser_language`, `ip_address`, `page_number`, `host_name`, `session_id`, `query_session_id`, `query_raw`, `device_type`, `geo_continent_name`, `geo_city_name`, `geo_country_iso_code`, `geo_region_name`, `geo_latitude`, `geo_longitude`, `click_link_url`, `click_link_index`, `click_clicked_object_id`, `click_title`, `click_query_id`, `click_context`, `query_type`, `query_total_banners`, `query_total_documents`, `query_response_time_ms`, `query_actual_query`, `query_quick_link_id`, `query_results_per_page`, `query_environment`, `query_request_type`, `query_banners`, `query_billable`, `application_name`, `application_version`, `http_status_code`, `aws_region`) FORMAT RowBinary ), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8b6cbba in /usr/bin/clickhouse | |
| 1. DB::Exception::Exception<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x8c0b8a3 in /usr/bin/clickhouse | |
| 2. DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0xff30579 in /usr/bin/clickhouse | |
| 3. DB::StorageReplicatedMergeTree::write(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>) @ 0xff3068a in /usr/bin/clickhouse | |
| 4. DB::PushingToViewsBlockOutputStream::PushingToViewsBlockOutputStream(std::__1::shared_ptr<DB::IStorage> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::IAST> const&, bool) @ 0xf8b85da in /usr/bin/clickhouse | |
| 5. DB::InterpreterInsertQuery::execute() @ 0xf8b2e2b in /usr/bin/clickhouse | |
| 6. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0x100a5ef5 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0x100a2aa9 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0x100a1cd4 in /usr/bin/clickhouse | |
| 9. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf8b9472 in /usr/bin/clickhouse | |
| 10. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf8c39ab in /usr/bin/clickhouse | |
| 11. DB::SquashingBlockOutputStream::finalize() @ 0xf8c402c in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf8c40c9 in /usr/bin/clickhouse | |
| 13. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xf332d1d in /usr/bin/clickhouse | |
| 14. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf8b6279 in /usr/bin/clickhouse | |
| 15. DB::IBlockInputStream::read() @ 0xf30f452 in /usr/bin/clickhouse | |
| 16. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xf332f76 in /usr/bin/clickhouse | |
| 17. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xfc0eaa5 in /usr/bin/clickhouse | |
| 18. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0x103c36bf in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x103c6e49 in /usr/bin/clickhouse | |
| 20. DB::HTTPServerConnection::run() @ 0x1044acd0 in /usr/bin/clickhouse | |
| 21. Poco::Net::TCPServerConnection::start() @ 0x12a3fd4f in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerDispatcher::run() @ 0x12a417da in /usr/bin/clickhouse | |
| 23. Poco::PooledThread::run() @ 0x12b7ab39 in /usr/bin/clickhouse | |
| 24. Poco::ThreadImpl::runnableEntry(void*) @ 0x12b76b2a in /usr/bin/clickhouse | |
| 25. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 26. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| 2021.06.24 08:39:32.090053 [ 49884 ] {895582f8-73ac-4fb3-8819-d3afb590f566} <Error> DynamicQueryHandler: Code: 242, e.displayText() = DB::Exception: Table is in readonly mode (zookeeper path: /clickhouse/analytics/tables/01/events_local), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8b6cbba in /usr/bin/clickhouse | |
| 1. DB::Exception::Exception<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x8c0b8a3 in /usr/bin/clickhouse | |
| 2. DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0xff30579 in /usr/bin/clickhouse | |
| 3. DB::StorageReplicatedMergeTree::write(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>) @ 0xff3068a in /usr/bin/clickhouse | |
| 4. DB::PushingToViewsBlockOutputStream::PushingToViewsBlockOutputStream(std::__1::shared_ptr<DB::IStorage> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::IAST> const&, bool) @ 0xf8b85da in /usr/bin/clickhouse | |
| 5. DB::InterpreterInsertQuery::execute() @ 0xf8b2e2b in /usr/bin/clickhouse | |
| 6. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0x100a5ef5 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0x100a2aa9 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0x100a1cd4 in /usr/bin/clickhouse | |
| 9. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf8b9472 in /usr/bin/clickhouse | |
| 10. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf8c39ab in /usr/bin/clickhouse | |
| 11. DB::SquashingBlockOutputStream::finalize() @ 0xf8c402c in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf8c40c9 in /usr/bin/clickhouse | |
| 13. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xf332d1d in /usr/bin/clickhouse | |
| 14. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf8b6279 in /usr/bin/clickhouse | |
| 15. DB::IBlockInputStream::read() @ 0xf30f452 in /usr/bin/clickhouse | |
| 16. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xf332f76 in /usr/bin/clickhouse | |
| 17. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xfc0eaa5 in /usr/bin/clickhouse | |
| 18. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0x103c36bf in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x103c6e49 in /usr/bin/clickhouse | |
| 20. DB::HTTPServerConnection::run() @ 0x1044acd0 in /usr/bin/clickhouse | |
| 21. Poco::Net::TCPServerConnection::start() @ 0x12a3fd4f in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerDispatcher::run() @ 0x12a417da in /usr/bin/clickhouse | |
| 23. Poco::PooledThread::run() @ 0x12b7ab39 in /usr/bin/clickhouse | |
| 24. Poco::ThreadImpl::runnableEntry(void*) @ 0x12b76b2a in /usr/bin/clickhouse | |
| 25. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 26. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.6.5.37 (official build)) | |
| 2021.06.24 08:39:32.897345 [ 49884 ] {438b49d4-748f-44c5-adc6-4dec6114f18f} <Warning> ClusterProxy::SelectStreamFactory: Local replica of shard 1 is stale (delay: 1624523972s.) | |
| 2021.06.24 08:39:33.347698 [ 49884 ] {49ddb706-e61d-4f15-af31-5c77266ac176} <Error> executeQuery: Code: 242, e.displayText() = DB::Exception: Table is in readonly mode (zookeeper path: /clickhouse/analytics/tables/01/events_local) (version 21.6.5.37 (official build)) (from 172.31.18.125:35428) (in query: INSERT INTO events (`id`, `event_type`, `date`, `customer_id`, `engine_id`, `query`, `display_width_px`, `display_height_px`, `user_agent`, `referrer_url`, `referrer_title`, `browser_language`, `ip_address`, `page_number`, `host_name`, `session_id`, `query_session_id`, `query_raw`, `device_type`, `geo_continent_name`, `geo_city_name`, `geo_country_iso_code`, `geo_region_name`, `geo_latitude`, `geo_longitude`, `click_link_url`, `click_link_index`, `click_clicked_object_id`, `click_title`, `click_query_id`, `click_context`, `query_type`, `query_total_banners`, `query_total_documents`, `query_response_time_ms`, `query_actual_query`, `query_quick_link_id`, `query_results_per_page`, `query_environment`, `query_request_type`, `query_banners`, `query_billable`, `application_name`, `application_version`, `http_status_code`, `aws_region`) FORMAT RowBinary ), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8b6cbba in /usr/bin/clickhouse | |
| 1. DB::Exception::Exception<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x8c0b8a3 in /usr/bin/clickhouse | |
| 2. DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0xff30579 in /usr/bin/clickhouse | |
| 3. DB::StorageReplicatedMergeTree::write(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>) @ 0xff3068a in /usr/bin/clickhouse | |
| 4. DB::PushingToViewsBlockOutputStream::PushingToViewsBlockOutputStream(std::__1::shared_ptr<DB::IStorage> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::IAST> const&, bool) @ 0xf8b85da in /usr/bin/clickhouse | |
| 5. DB::InterpreterInsertQuery::execute() @ 0xf8b2e2b in /usr/bin/clickhouse | |
| 6. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0x100a5ef5 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0x100a2aa9 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0x100a1cd4 in /usr/bin/clickhouse | |
| 9. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf8b9472 in /usr/bin/clickhouse | |
| 10. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf8c39ab in /usr/bin/clickhouse | |
| 11. DB::SquashingBlockOutputStream::finalize() @ 0xf8c402c in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf8c40c9 in /usr/bin/clickhouse | |
| 13. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xf332d1d in /usr/bin/clickhouse | |
| 14. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf8b6279 in /usr/bin/clickhouse | |
| 15. DB::IBlockInputStream::read() @ 0xf30f452 in /usr/bin/clickhouse | |
| 16. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xf332f76 in /usr/bin/clickhouse | |
| 17. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xfc0eaa5 in /usr/bin/clickhouse | |
| 18. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0x103c36bf in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x103c6e49 in /usr/bin/clickhouse | |
| 20. DB::HTTPServerConnection::run() @ 0x1044acd0 in /usr/bin/clickhouse | |
| 21. Poco::Net::TCPServerConnection::start() @ 0x12a3fd4f in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerDispatcher::run() @ 0x12a417da in /usr/bin/clickhouse | |
| 23. Poco::PooledThread::run() @ 0x12b7ab39 in /usr/bin/clickhouse | |
| 24. Poco::ThreadImpl::runnableEntry(void*) @ 0x12b76b2a in /usr/bin/clickhouse | |
| 25. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 26. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| 2021.06.24 08:39:33.347876 [ 49884 ] {49ddb706-e61d-4f15-af31-5c77266ac176} <Error> DynamicQueryHandler: Code: 242, e.displayText() = DB::Exception: Table is in readonly mode (zookeeper path: /clickhouse/analytics/tables/01/events_local), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8b6cbba in /usr/bin/clickhouse | |
| 1. DB::Exception::Exception<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x8c0b8a3 in /usr/bin/clickhouse | |
| 2. DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0xff30579 in /usr/bin/clickhouse | |
| 3. DB::StorageReplicatedMergeTree::write(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>) @ 0xff3068a in /usr/bin/clickhouse | |
| 4. DB::PushingToViewsBlockOutputStream::PushingToViewsBlockOutputStream(std::__1::shared_ptr<DB::IStorage> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::IAST> const&, bool) @ 0xf8b85da in /usr/bin/clickhouse | |
| 5. DB::InterpreterInsertQuery::execute() @ 0xf8b2e2b in /usr/bin/clickhouse | |
| 6. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0x100a5ef5 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0x100a2aa9 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0x100a1cd4 in /usr/bin/clickhouse | |
| 9. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf8b9472 in /usr/bin/clickhouse | |
| 10. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf8c39ab in /usr/bin/clickhouse | |
| 11. DB::SquashingBlockOutputStream::finalize() @ 0xf8c402c in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf8c40c9 in /usr/bin/clickhouse | |
| 13. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xf332d1d in /usr/bin/clickhouse | |
| 14. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf8b6279 in /usr/bin/clickhouse | |
| 15. DB::IBlockInputStream::read() @ 0xf30f452 in /usr/bin/clickhouse | |
| 16. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xf332f76 in /usr/bin/clickhouse | |
| 17. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xfc0eaa5 in /usr/bin/clickhouse | |
| 18. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0x103c36bf in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x103c6e49 in /usr/bin/clickhouse | |
| 20. DB::HTTPServerConnection::run() @ 0x1044acd0 in /usr/bin/clickhouse | |
| 21. Poco::Net::TCPServerConnection::start() @ 0x12a3fd4f in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerDispatcher::run() @ 0x12a417da in /usr/bin/clickhouse | |
| 23. Poco::PooledThread::run() @ 0x12b7ab39 in /usr/bin/clickhouse | |
| 24. Poco::ThreadImpl::runnableEntry(void*) @ 0x12b76b2a in /usr/bin/clickhouse | |
| 25. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 26. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.6.5.37 (official build)) | |
| 2021.06.24 08:39:33.990556 [ 49884 ] {f64966b3-a9fe-43fc-9cbc-0e631bfe5d25} <Warning> ClusterProxy::SelectStreamFactory: Local replica of shard 1 is stale (delay: 1624523973s.) | |
| 2021.06.24 08:39:34.032994 [ 49884 ] {136e7633-673e-4ffe-8bd4-b34dadd9f3c7} <Error> executeQuery: Code: 242, e.displayText() = DB::Exception: Table is in readonly mode (zookeeper path: /clickhouse/analytics/tables/01/events_local) (version 21.6.5.37 (official build)) (from 172.31.18.125:35440) (in query: INSERT INTO events (`id`, `event_type`, `date`, `customer_id`, `engine_id`, `query`, `display_width_px`, `display_height_px`, `user_agent`, `referrer_url`, `referrer_title`, `browser_language`, `ip_address`, `page_number`, `host_name`, `session_id`, `query_session_id`, `query_raw`, `device_type`, `geo_continent_name`, `geo_city_name`, `geo_country_iso_code`, `geo_region_name`, `geo_latitude`, `geo_longitude`, `click_link_url`, `click_link_index`, `click_clicked_object_id`, `click_title`, `click_query_id`, `click_context`, `query_type`, `query_total_banners`, `query_total_documents`, `query_response_time_ms`, `query_actual_query`, `query_quick_link_id`, `query_results_per_page`, `query_environment`, `query_request_type`, `query_banners`, `query_billable`, `application_name`, `application_version`, `http_status_code`, `aws_region`) FORMAT RowBinary ), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8b6cbba in /usr/bin/clickhouse | |
| 1. DB::Exception::Exception<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x8c0b8a3 in /usr/bin/clickhouse | |
| 2. DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0xff30579 in /usr/bin/clickhouse | |
| 3. DB::StorageReplicatedMergeTree::write(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>) @ 0xff3068a in /usr/bin/clickhouse | |
| 4. DB::PushingToViewsBlockOutputStream::PushingToViewsBlockOutputStream(std::__1::shared_ptr<DB::IStorage> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::IAST> const&, bool) @ 0xf8b85da in /usr/bin/clickhouse | |
| 5. DB::InterpreterInsertQuery::execute() @ 0xf8b2e2b in /usr/bin/clickhouse | |
| 6. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0x100a5ef5 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0x100a2aa9 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0x100a1cd4 in /usr/bin/clickhouse | |
| 9. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf8b9472 in /usr/bin/clickhouse | |
| 10. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf8c39ab in /usr/bin/clickhouse | |
| 11. DB::SquashingBlockOutputStream::finalize() @ 0xf8c402c in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf8c40c9 in /usr/bin/clickhouse | |
| 13. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xf332d1d in /usr/bin/clickhouse | |
| 14. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf8b6279 in /usr/bin/clickhouse | |
| 15. DB::IBlockInputStream::read() @ 0xf30f452 in /usr/bin/clickhouse | |
| 16. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xf332f76 in /usr/bin/clickhouse | |
| 17. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xfc0eaa5 in /usr/bin/clickhouse | |
| 18. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0x103c36bf in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x103c6e49 in /usr/bin/clickhouse | |
| 20. DB::HTTPServerConnection::run() @ 0x1044acd0 in /usr/bin/clickhouse | |
| 21. Poco::Net::TCPServerConnection::start() @ 0x12a3fd4f in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerDispatcher::run() @ 0x12a417da in /usr/bin/clickhouse | |
| 23. Poco::PooledThread::run() @ 0x12b7ab39 in /usr/bin/clickhouse | |
| 24. Poco::ThreadImpl::runnableEntry(void*) @ 0x12b76b2a in /usr/bin/clickhouse | |
| 25. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 26. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| 2021.06.24 08:39:34.033175 [ 49884 ] {136e7633-673e-4ffe-8bd4-b34dadd9f3c7} <Error> DynamicQueryHandler: Code: 242, e.displayText() = DB::Exception: Table is in readonly mode (zookeeper path: /clickhouse/analytics/tables/01/events_local), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8b6cbba in /usr/bin/clickhouse | |
| 1. DB::Exception::Exception<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x8c0b8a3 in /usr/bin/clickhouse | |
| 2. DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0xff30579 in /usr/bin/clickhouse | |
| 3. DB::StorageReplicatedMergeTree::write(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>) @ 0xff3068a in /usr/bin/clickhouse | |
| 4. DB::PushingToViewsBlockOutputStream::PushingToViewsBlockOutputStream(std::__1::shared_ptr<DB::IStorage> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::IAST> const&, bool) @ 0xf8b85da in /usr/bin/clickhouse | |
| 5. DB::InterpreterInsertQuery::execute() @ 0xf8b2e2b in /usr/bin/clickhouse | |
| 6. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0x100a5ef5 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0x100a2aa9 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0x100a1cd4 in /usr/bin/clickhouse | |
| 9. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf8b9472 in /usr/bin/clickhouse | |
| 10. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf8c39ab in /usr/bin/clickhouse | |
| 11. DB::SquashingBlockOutputStream::finalize() @ 0xf8c402c in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf8c40c9 in /usr/bin/clickhouse | |
| 13. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xf332d1d in /usr/bin/clickhouse | |
| 14. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf8b6279 in /usr/bin/clickhouse | |
| 15. DB::IBlockInputStream::read() @ 0xf30f452 in /usr/bin/clickhouse | |
| 16. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xf332f76 in /usr/bin/clickhouse | |
| 17. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xfc0eaa5 in /usr/bin/clickhouse | |
| 18. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0x103c36bf in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x103c6e49 in /usr/bin/clickhouse | |
| 20. DB::HTTPServerConnection::run() @ 0x1044acd0 in /usr/bin/clickhouse | |
| 21. Poco::Net::TCPServerConnection::start() @ 0x12a3fd4f in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerDispatcher::run() @ 0x12a417da in /usr/bin/clickhouse | |
| 23. Poco::PooledThread::run() @ 0x12b7ab39 in /usr/bin/clickhouse | |
| 24. Poco::ThreadImpl::runnableEntry(void*) @ 0x12b76b2a in /usr/bin/clickhouse | |
| 25. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 26. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.6.5.37 (official build)) | |
| 2021.06.24 08:39:34.354695 [ 49884 ] {e9b2e310-24a3-40b3-bdeb-61c7cb862386} <Warning> ClusterProxy::SelectStreamFactory: Local replica of shard 1 is stale (delay: 1624523974s.) | |
| 2021.06.24 08:39:34.387797 [ 49884 ] {f9c1467a-5daf-476a-ac32-ab57ed6c32c5} <Error> executeQuery: Code: 242, e.displayText() = DB::Exception: Table is in readonly mode (zookeeper path: /clickhouse/analytics/tables/01/events_local) (version 21.6.5.37 (official build)) (from 172.31.18.125:35452) (in query: INSERT INTO events (`id`, `event_type`, `date`, `customer_id`, `engine_id`, `query`, `display_width_px`, `display_height_px`, `user_agent`, `referrer_url`, `referrer_title`, `browser_language`, `ip_address`, `page_number`, `host_name`, `session_id`, `query_session_id`, `query_raw`, `device_type`, `geo_continent_name`, `geo_city_name`, `geo_country_iso_code`, `geo_region_name`, `geo_latitude`, `geo_longitude`, `click_link_url`, `click_link_index`, `click_clicked_object_id`, `click_title`, `click_query_id`, `click_context`, `query_type`, `query_total_banners`, `query_total_documents`, `query_response_time_ms`, `query_actual_query`, `query_quick_link_id`, `query_results_per_page`, `query_environment`, `query_request_type`, `query_banners`, `query_billable`, `application_name`, `application_version`, `http_status_code`, `aws_region`) FORMAT RowBinary ), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8b6cbba in /usr/bin/clickhouse | |
| 1. DB::Exception::Exception<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x8c0b8a3 in /usr/bin/clickhouse | |
| 2. DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0xff30579 in /usr/bin/clickhouse | |
| 3. DB::StorageReplicatedMergeTree::write(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>) @ 0xff3068a in /usr/bin/clickhouse | |
| 4. DB::PushingToViewsBlockOutputStream::PushingToViewsBlockOutputStream(std::__1::shared_ptr<DB::IStorage> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::IAST> const&, bool) @ 0xf8b85da in /usr/bin/clickhouse | |
| 5. DB::InterpreterInsertQuery::execute() @ 0xf8b2e2b in /usr/bin/clickhouse | |
| 6. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0x100a5ef5 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0x100a2aa9 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0x100a1cd4 in /usr/bin/clickhouse | |
| 9. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf8b9472 in /usr/bin/clickhouse | |
| 10. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf8c39ab in /usr/bin/clickhouse | |
| 11. DB::SquashingBlockOutputStream::finalize() @ 0xf8c402c in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf8c40c9 in /usr/bin/clickhouse | |
| 13. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xf332d1d in /usr/bin/clickhouse | |
| 14. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf8b6279 in /usr/bin/clickhouse | |
| 15. DB::IBlockInputStream::read() @ 0xf30f452 in /usr/bin/clickhouse | |
| 16. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xf332f76 in /usr/bin/clickhouse | |
| 17. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xfc0eaa5 in /usr/bin/clickhouse | |
| 18. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0x103c36bf in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x103c6e49 in /usr/bin/clickhouse | |
| 20. DB::HTTPServerConnection::run() @ 0x1044acd0 in /usr/bin/clickhouse | |
| 21. Poco::Net::TCPServerConnection::start() @ 0x12a3fd4f in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerDispatcher::run() @ 0x12a417da in /usr/bin/clickhouse | |
| 23. Poco::PooledThread::run() @ 0x12b7ab39 in /usr/bin/clickhouse | |
| 24. Poco::ThreadImpl::runnableEntry(void*) @ 0x12b76b2a in /usr/bin/clickhouse | |
| 25. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 26. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| 2021.06.24 08:39:34.387978 [ 49884 ] {f9c1467a-5daf-476a-ac32-ab57ed6c32c5} <Error> DynamicQueryHandler: Code: 242, e.displayText() = DB::Exception: Table is in readonly mode (zookeeper path: /clickhouse/analytics/tables/01/events_local), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8b6cbba in /usr/bin/clickhouse | |
| 1. DB::Exception::Exception<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x8c0b8a3 in /usr/bin/clickhouse | |
| 2. DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0xff30579 in /usr/bin/clickhouse | |
| 3. DB::StorageReplicatedMergeTree::write(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>) @ 0xff3068a in /usr/bin/clickhouse | |
| 4. DB::PushingToViewsBlockOutputStream::PushingToViewsBlockOutputStream(std::__1::shared_ptr<DB::IStorage> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::IAST> const&, bool) @ 0xf8b85da in /usr/bin/clickhouse | |
| 5. DB::InterpreterInsertQuery::execute() @ 0xf8b2e2b in /usr/bin/clickhouse | |
| 6. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0x100a5ef5 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0x100a2aa9 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0x100a1cd4 in /usr/bin/clickhouse | |
| 9. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf8b9472 in /usr/bin/clickhouse | |
| 10. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf8c39ab in /usr/bin/clickhouse | |
| 11. DB::SquashingBlockOutputStream::finalize() @ 0xf8c402c in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf8c40c9 in /usr/bin/clickhouse | |
| 13. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xf332d1d in /usr/bin/clickhouse | |
| 14. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf8b6279 in /usr/bin/clickhouse | |
| 15. DB::IBlockInputStream::read() @ 0xf30f452 in /usr/bin/clickhouse | |
| 16. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xf332f76 in /usr/bin/clickhouse | |
| 17. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xfc0eaa5 in /usr/bin/clickhouse | |
| 18. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0x103c36bf in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x103c6e49 in /usr/bin/clickhouse | |
| 20. DB::HTTPServerConnection::run() @ 0x1044acd0 in /usr/bin/clickhouse | |
| 21. Poco::Net::TCPServerConnection::start() @ 0x12a3fd4f in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerDispatcher::run() @ 0x12a417da in /usr/bin/clickhouse | |
| 23. Poco::PooledThread::run() @ 0x12b7ab39 in /usr/bin/clickhouse | |
| 24. Poco::ThreadImpl::runnableEntry(void*) @ 0x12b76b2a in /usr/bin/clickhouse | |
| 25. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 26. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.6.5.37 (official build)) | |
| 2021.06.24 08:39:35.041336 [ 49884 ] {fd460ccf-d05b-4f88-a3e8-ce161f01162a} <Warning> ClusterProxy::SelectStreamFactory: Local replica of shard 1 is stale (delay: 1624523975s.) | |
| 2021.06.24 08:39:35.603946 [ 49884 ] {ba1b9f4f-f526-41a8-92c4-9d3993683335} <Error> executeQuery: Code: 242, e.displayText() = DB::Exception: Table is in readonly mode (zookeeper path: /clickhouse/analytics/tables/01/events_local) (version 21.6.5.37 (official build)) (from 172.31.18.125:35464) (in query: INSERT INTO events (`id`, `event_type`, `date`, `customer_id`, `engine_id`, `query`, `display_width_px`, `display_height_px`, `user_agent`, `referrer_url`, `referrer_title`, `browser_language`, `ip_address`, `page_number`, `host_name`, `session_id`, `query_session_id`, `query_raw`, `device_type`, `geo_continent_name`, `geo_city_name`, `geo_country_iso_code`, `geo_region_name`, `geo_latitude`, `geo_longitude`, `click_link_url`, `click_link_index`, `click_clicked_object_id`, `click_title`, `click_query_id`, `click_context`, `query_type`, `query_total_banners`, `query_total_documents`, `query_response_time_ms`, `query_actual_query`, `query_quick_link_id`, `query_results_per_page`, `query_environment`, `query_request_type`, `query_banners`, `query_billable`, `application_name`, `application_version`, `http_status_code`, `aws_region`) FORMAT RowBinary ), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8b6cbba in /usr/bin/clickhouse | |
| 1. DB::Exception::Exception<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x8c0b8a3 in /usr/bin/clickhouse | |
| 2. DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0xff30579 in /usr/bin/clickhouse | |
| 3. DB::StorageReplicatedMergeTree::write(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>) @ 0xff3068a in /usr/bin/clickhouse | |
| 4. DB::PushingToViewsBlockOutputStream::PushingToViewsBlockOutputStream(std::__1::shared_ptr<DB::IStorage> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::IAST> const&, bool) @ 0xf8b85da in /usr/bin/clickhouse | |
| 5. DB::InterpreterInsertQuery::execute() @ 0xf8b2e2b in /usr/bin/clickhouse | |
| 6. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0x100a5ef5 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0x100a2aa9 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0x100a1cd4 in /usr/bin/clickhouse | |
| 9. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf8b9472 in /usr/bin/clickhouse | |
| 10. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf8c39ab in /usr/bin/clickhouse | |
| 11. DB::SquashingBlockOutputStream::finalize() @ 0xf8c402c in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf8c40c9 in /usr/bin/clickhouse | |
| 13. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xf332d1d in /usr/bin/clickhouse | |
| 14. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf8b6279 in /usr/bin/clickhouse | |
| 15. DB::IBlockInputStream::read() @ 0xf30f452 in /usr/bin/clickhouse | |
| 16. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xf332f76 in /usr/bin/clickhouse | |
| 17. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xfc0eaa5 in /usr/bin/clickhouse | |
| 18. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0x103c36bf in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x103c6e49 in /usr/bin/clickhouse | |
| 20. DB::HTTPServerConnection::run() @ 0x1044acd0 in /usr/bin/clickhouse | |
| 21. Poco::Net::TCPServerConnection::start() @ 0x12a3fd4f in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerDispatcher::run() @ 0x12a417da in /usr/bin/clickhouse | |
| 23. Poco::PooledThread::run() @ 0x12b7ab39 in /usr/bin/clickhouse | |
| 24. Poco::ThreadImpl::runnableEntry(void*) @ 0x12b76b2a in /usr/bin/clickhouse | |
| 25. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 26. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| 2021.06.24 08:39:35.604258 [ 49884 ] {ba1b9f4f-f526-41a8-92c4-9d3993683335} <Error> DynamicQueryHandler: Code: 242, e.displayText() = DB::Exception: Table is in readonly mode (zookeeper path: /clickhouse/analytics/tables/01/events_local), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8b6cbba in /usr/bin/clickhouse | |
| 1. DB::Exception::Exception<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x8c0b8a3 in /usr/bin/clickhouse | |
| 2. DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0xff30579 in /usr/bin/clickhouse | |
| 3. DB::StorageReplicatedMergeTree::write(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>) @ 0xff3068a in /usr/bin/clickhouse | |
| 4. DB::PushingToViewsBlockOutputStream::PushingToViewsBlockOutputStream(std::__1::shared_ptr<DB::IStorage> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::IAST> const&, bool) @ 0xf8b85da in /usr/bin/clickhouse | |
| 5. DB::InterpreterInsertQuery::execute() @ 0xf8b2e2b in /usr/bin/clickhouse | |
| 6. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0x100a5ef5 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0x100a2aa9 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0x100a1cd4 in /usr/bin/clickhouse | |
| 9. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf8b9472 in /usr/bin/clickhouse | |
| 10. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf8c39ab in /usr/bin/clickhouse | |
| 11. DB::SquashingBlockOutputStream::finalize() @ 0xf8c402c in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf8c40c9 in /usr/bin/clickhouse | |
| 13. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xf332d1d in /usr/bin/clickhouse | |
| 14. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf8b6279 in /usr/bin/clickhouse | |
| 15. DB::IBlockInputStream::read() @ 0xf30f452 in /usr/bin/clickhouse | |
| 16. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xf332f76 in /usr/bin/clickhouse | |
| 17. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xfc0eaa5 in /usr/bin/clickhouse | |
| 18. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0x103c36bf in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x103c6e49 in /usr/bin/clickhouse | |
| 20. DB::HTTPServerConnection::run() @ 0x1044acd0 in /usr/bin/clickhouse | |
| 21. Poco::Net::TCPServerConnection::start() @ 0x12a3fd4f in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerDispatcher::run() @ 0x12a417da in /usr/bin/clickhouse | |
| 23. Poco::PooledThread::run() @ 0x12b7ab39 in /usr/bin/clickhouse | |
| 24. Poco::ThreadImpl::runnableEntry(void*) @ 0x12b76b2a in /usr/bin/clickhouse | |
| 25. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 26. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.6.5.37 (official build)) | |
| 2021.06.24 08:39:36.073026 [ 49884 ] {993ac219-3cdc-479c-84af-ddeae4bc63a2} <Warning> ClusterProxy::SelectStreamFactory: Local replica of shard 1 is stale (delay: 1624523976s.) | |
| 2021.06.24 08:39:36.872375 [ 49884 ] {69aa4783-de4a-41ba-bfc4-d89c4bf7a925} <Error> executeQuery: Code: 242, e.displayText() = DB::Exception: Table is in readonly mode (zookeeper path: /clickhouse/analytics/tables/01/events_local) (version 21.6.5.37 (official build)) (from 172.31.18.125:35476) (in query: INSERT INTO events (`id`, `event_type`, `date`, `customer_id`, `engine_id`, `query`, `display_width_px`, `display_height_px`, `user_agent`, `referrer_url`, `referrer_title`, `browser_language`, `ip_address`, `page_number`, `host_name`, `session_id`, `query_session_id`, `query_raw`, `device_type`, `geo_continent_name`, `geo_city_name`, `geo_country_iso_code`, `geo_region_name`, `geo_latitude`, `geo_longitude`, `click_link_url`, `click_link_index`, `click_clicked_object_id`, `click_title`, `click_query_id`, `click_context`, `query_type`, `query_total_banners`, `query_total_documents`, `query_response_time_ms`, `query_actual_query`, `query_quick_link_id`, `query_results_per_page`, `query_environment`, `query_request_type`, `query_banners`, `query_billable`, `application_name`, `application_version`, `http_status_code`, `aws_region`) FORMAT RowBinary ), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8b6cbba in /usr/bin/clickhouse | |
| 1. DB::Exception::Exception<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x8c0b8a3 in /usr/bin/clickhouse | |
| 2. DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0xff30579 in /usr/bin/clickhouse | |
| 3. DB::StorageReplicatedMergeTree::write(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>) @ 0xff3068a in /usr/bin/clickhouse | |
| 4. DB::PushingToViewsBlockOutputStream::PushingToViewsBlockOutputStream(std::__1::shared_ptr<DB::IStorage> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::IAST> const&, bool) @ 0xf8b85da in /usr/bin/clickhouse | |
| 5. DB::InterpreterInsertQuery::execute() @ 0xf8b2e2b in /usr/bin/clickhouse | |
| 6. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0x100a5ef5 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0x100a2aa9 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0x100a1cd4 in /usr/bin/clickhouse | |
| 9. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf8b9472 in /usr/bin/clickhouse | |
| 10. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf8c39ab in /usr/bin/clickhouse | |
| 11. DB::SquashingBlockOutputStream::finalize() @ 0xf8c402c in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf8c40c9 in /usr/bin/clickhouse | |
| 13. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xf332d1d in /usr/bin/clickhouse | |
| 14. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf8b6279 in /usr/bin/clickhouse | |
| 15. DB::IBlockInputStream::read() @ 0xf30f452 in /usr/bin/clickhouse | |
| 16. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xf332f76 in /usr/bin/clickhouse | |
| 17. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xfc0eaa5 in /usr/bin/clickhouse | |
| 18. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0x103c36bf in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x103c6e49 in /usr/bin/clickhouse | |
| 20. DB::HTTPServerConnection::run() @ 0x1044acd0 in /usr/bin/clickhouse | |
| 21. Poco::Net::TCPServerConnection::start() @ 0x12a3fd4f in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerDispatcher::run() @ 0x12a417da in /usr/bin/clickhouse | |
| 23. Poco::PooledThread::run() @ 0x12b7ab39 in /usr/bin/clickhouse | |
| 24. Poco::ThreadImpl::runnableEntry(void*) @ 0x12b76b2a in /usr/bin/clickhouse | |
| 25. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 26. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| 2021.06.24 08:39:36.872679 [ 49884 ] {69aa4783-de4a-41ba-bfc4-d89c4bf7a925} <Error> DynamicQueryHandler: Code: 242, e.displayText() = DB::Exception: Table is in readonly mode (zookeeper path: /clickhouse/analytics/tables/01/events_local), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8b6cbba in /usr/bin/clickhouse | |
| 1. DB::Exception::Exception<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x8c0b8a3 in /usr/bin/clickhouse | |
| 2. DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0xff30579 in /usr/bin/clickhouse | |
| 3. DB::StorageReplicatedMergeTree::write(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>) @ 0xff3068a in /usr/bin/clickhouse | |
| 4. DB::PushingToViewsBlockOutputStream::PushingToViewsBlockOutputStream(std::__1::shared_ptr<DB::IStorage> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::IAST> const&, bool) @ 0xf8b85da in /usr/bin/clickhouse | |
| 5. DB::InterpreterInsertQuery::execute() @ 0xf8b2e2b in /usr/bin/clickhouse | |
| 6. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0x100a5ef5 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0x100a2aa9 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0x100a1cd4 in /usr/bin/clickhouse | |
| 9. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf8b9472 in /usr/bin/clickhouse | |
| 10. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf8c39ab in /usr/bin/clickhouse | |
| 11. DB::SquashingBlockOutputStream::finalize() @ 0xf8c402c in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf8c40c9 in /usr/bin/clickhouse | |
| 13. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xf332d1d in /usr/bin/clickhouse | |
| 14. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf8b6279 in /usr/bin/clickhouse | |
| 15. DB::IBlockInputStream::read() @ 0xf30f452 in /usr/bin/clickhouse | |
| 16. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xf332f76 in /usr/bin/clickhouse | |
| 17. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xfc0eaa5 in /usr/bin/clickhouse | |
| 18. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0x103c36bf in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x103c6e49 in /usr/bin/clickhouse | |
| 20. DB::HTTPServerConnection::run() @ 0x1044acd0 in /usr/bin/clickhouse | |
| 21. Poco::Net::TCPServerConnection::start() @ 0x12a3fd4f in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerDispatcher::run() @ 0x12a417da in /usr/bin/clickhouse | |
| 23. Poco::PooledThread::run() @ 0x12b7ab39 in /usr/bin/clickhouse | |
| 24. Poco::ThreadImpl::runnableEntry(void*) @ 0x12b76b2a in /usr/bin/clickhouse | |
| 25. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 26. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.6.5.37 (official build)) | |
| 2021.06.24 08:39:37.132061 [ 49884 ] {0d5b9a0a-3742-4bee-b747-27406ca4a59a} <Error> executeQuery: Code: 242, e.displayText() = DB::Exception: Table is in readonly mode (zookeeper path: /clickhouse/analytics/tables/01/events_local) (version 21.6.5.37 (official build)) (from 172.31.18.125:35484) (in query: INSERT INTO events (`id`, `event_type`, `date`, `customer_id`, `engine_id`, `query`, `display_width_px`, `display_height_px`, `user_agent`, `referrer_url`, `referrer_title`, `browser_language`, `ip_address`, `page_number`, `host_name`, `session_id`, `query_session_id`, `query_raw`, `device_type`, `geo_continent_name`, `geo_city_name`, `geo_country_iso_code`, `geo_region_name`, `geo_latitude`, `geo_longitude`, `click_link_url`, `click_link_index`, `click_clicked_object_id`, `click_title`, `click_query_id`, `click_context`, `query_type`, `query_total_banners`, `query_total_documents`, `query_response_time_ms`, `query_actual_query`, `query_quick_link_id`, `query_results_per_page`, `query_environment`, `query_request_type`, `query_banners`, `query_billable`, `application_name`, `application_version`, `http_status_code`, `aws_region`) FORMAT RowBinary ), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8b6cbba in /usr/bin/clickhouse | |
| 1. DB::Exception::Exception<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x8c0b8a3 in /usr/bin/clickhouse | |
| 2. DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0xff30579 in /usr/bin/clickhouse | |
| 3. DB::StorageReplicatedMergeTree::write(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>) @ 0xff3068a in /usr/bin/clickhouse | |
| 4. DB::PushingToViewsBlockOutputStream::PushingToViewsBlockOutputStream(std::__1::shared_ptr<DB::IStorage> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::IAST> const&, bool) @ 0xf8b85da in /usr/bin/clickhouse | |
| 5. DB::InterpreterInsertQuery::execute() @ 0xf8b2e2b in /usr/bin/clickhouse | |
| 6. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0x100a5ef5 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0x100a2aa9 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0x100a1cd4 in /usr/bin/clickhouse | |
| 9. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf8b9472 in /usr/bin/clickhouse | |
| 10. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf8c39ab in /usr/bin/clickhouse | |
| 11. DB::SquashingBlockOutputStream::finalize() @ 0xf8c402c in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf8c40c9 in /usr/bin/clickhouse | |
| 13. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xf332d1d in /usr/bin/clickhouse | |
| 14. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf8b6279 in /usr/bin/clickhouse | |
| 15. DB::IBlockInputStream::read() @ 0xf30f452 in /usr/bin/clickhouse | |
| 16. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xf332f76 in /usr/bin/clickhouse | |
| 17. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xfc0eaa5 in /usr/bin/clickhouse | |
| 18. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0x103c36bf in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x103c6e49 in /usr/bin/clickhouse | |
| 20. DB::HTTPServerConnection::run() @ 0x1044acd0 in /usr/bin/clickhouse | |
| 21. Poco::Net::TCPServerConnection::start() @ 0x12a3fd4f in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerDispatcher::run() @ 0x12a417da in /usr/bin/clickhouse | |
| 23. Poco::PooledThread::run() @ 0x12b7ab39 in /usr/bin/clickhouse | |
| 24. Poco::ThreadImpl::runnableEntry(void*) @ 0x12b76b2a in /usr/bin/clickhouse | |
| 25. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 26. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| 2021.06.24 08:39:37.132372 [ 49884 ] {0d5b9a0a-3742-4bee-b747-27406ca4a59a} <Error> DynamicQueryHandler: Code: 242, e.displayText() = DB::Exception: Table is in readonly mode (zookeeper path: /clickhouse/analytics/tables/01/events_local), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8b6cbba in /usr/bin/clickhouse | |
| 1. DB::Exception::Exception<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x8c0b8a3 in /usr/bin/clickhouse | |
| 2. DB::StorageReplicatedMergeTree::assertNotReadonly() const @ 0xff30579 in /usr/bin/clickhouse | |
| 3. DB::StorageReplicatedMergeTree::write(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>) @ 0xff3068a in /usr/bin/clickhouse | |
| 4. DB::PushingToViewsBlockOutputStream::PushingToViewsBlockOutputStream(std::__1::shared_ptr<DB::IStorage> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::IAST> const&, bool) @ 0xf8b85da in /usr/bin/clickhouse | |
| 5. DB::InterpreterInsertQuery::execute() @ 0xf8b2e2b in /usr/bin/clickhouse | |
| 6. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0x100a5ef5 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0x100a2aa9 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0x100a1cd4 in /usr/bin/clickhouse | |
| 9. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf8b9472 in /usr/bin/clickhouse | |
| 10. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf8c39ab in /usr/bin/clickhouse | |
| 11. DB::SquashingBlockOutputStream::finalize() @ 0xf8c402c in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf8c40c9 in /usr/bin/clickhouse | |
| 13. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xf332d1d in /usr/bin/clickhouse | |
| 14. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf8b6279 in /usr/bin/clickhouse | |
| 15. DB::IBlockInputStream::read() @ 0xf30f452 in /usr/bin/clickhouse | |
| 16. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xf332f76 in /usr/bin/clickhouse | |
| 17. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xfc0eaa5 in /usr/bin/clickhouse | |
| 18. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0x103c36bf in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x103c6e49 in /usr/bin/clickhouse | |
| 20. DB::HTTPServerConnection::run() @ 0x1044acd0 in /usr/bin/clickhouse | |
| 21. Poco::Net::TCPServerConnection::start() @ 0x12a3fd4f in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerDispatcher::run() @ 0x12a417da in /usr/bin/clickhouse | |
| 23. Poco::PooledThread::run() @ 0x12b7ab39 in /usr/bin/clickhouse | |
| 24. Poco::ThreadImpl::runnableEntry(void*) @ 0x12b76b2a in /usr/bin/clickhouse | |
| 25. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 26. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.6.5.37 (official build)) | |
| 2021.06.24 08:39:37.922828 [ 49884 ] {32af227c-dabe-49d9-bf07-1bbeb20bfb7f} <Warning> ClusterProxy::SelectStreamFactory: Local replica of shard 1 is stale (delay: 1624523977s.) | |
| 2021.06.24 08:39:54.616759 [ 49884 ] {fe9383b3-6d35-437a-a629-ae548bf56c59} <Error> executeQuery: Code: 62, e.displayText() = DB::Exception: Syntax error: failed at position 74 ('?') (line 2, col 20): ? AND query= ?. Expected one of: EXTRACT, TIMESTAMPADD, TIMESTAMPDIFF, TIMESTAMP_DIFF, TRIM expression, number, substitution, DATE_ADD, DoubleColon, RIGHT, tuple, TIMESTAMP, literal, qualified asterisk, SELECT subquery, DATE_ADD expression, asterisk, EXTRACT expression, DATE, NULL, compound identifier, string concatenation expression, identifier, MySQL-style global variable, array element expression, TIMESTAMP operator expression, element of expression, expression with prefix unary operator, DATE operator expression, list, delimited by operator of variable arity, DATE_DIFF, parenthesized expression, COLUMNS matcher, RIGHT expression, array, TRIM, DATE_DIFF expression, case, string literal, unary minus expression, INTERVAL operator expression, DATESUB, TIMESTAMPSUB, CAST AS expression, INTERVAL, CASE, LEFT expression, LEFT, CAST operator, CAST, additive expression, SUBSTRING, list of elements, RTRIM, DATEDIFF, SUBSTRING expression, BETWEEN expression, function, DATEADD, DATE_SUB, tuple element expression, collection of literals, TIMESTAMP_ADD, list, delimited by binary operators, CAST expression, token, COLUMNS, TIMESTAMP_SUB, LTRIM, multiplicative expression (version 21.6.5.37 (official build)) (from 172.31.18.125:35654) (in query: ALTER TABLE events_local ON CLUSTER analytics DELETE WHERE customer_id= ? AND query= ?), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8b6cbba in /usr/bin/clickhouse | |
| 1. DB::parseQueryAndMovePosition(DB::IParser&, char const*&, char const*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, bool, unsigned long, unsigned long) @ 0x1234b208 in /usr/bin/clickhouse | |
| 2. ? @ 0xfc08f48 in /usr/bin/clickhouse | |
| 3. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xfc0dfd5 in /usr/bin/clickhouse | |
| 4. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0x103c36bf in /usr/bin/clickhouse | |
| 5. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x103c6e49 in /usr/bin/clickhouse | |
| 6. DB::HTTPServerConnection::run() @ 0x1044acd0 in /usr/bin/clickhouse | |
| 7. Poco::Net::TCPServerConnection::start() @ 0x12a3fd4f in /usr/bin/clickhouse | |
| 8. Poco::Net::TCPServerDispatcher::run() @ 0x12a417da in /usr/bin/clickhouse | |
| 9. Poco::PooledThread::run() @ 0x12b7ab39 in /usr/bin/clickhouse | |
| 10. Poco::ThreadImpl::runnableEntry(void*) @ 0x12b76b2a in /usr/bin/clickhouse | |
| 11. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 12. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| 2021.06.24 08:39:54.616888 [ 49884 ] {fe9383b3-6d35-437a-a629-ae548bf56c59} <Error> DynamicQueryHandler: Code: 62, e.displayText() = DB::Exception: Syntax error: failed at position 74 ('?') (line 2, col 20): ? AND query= ?. Expected one of: EXTRACT, TIMESTAMPADD, TIMESTAMPDIFF, TIMESTAMP_DIFF, TRIM expression, number, substitution, DATE_ADD, DoubleColon, RIGHT, tuple, TIMESTAMP, literal, qualified asterisk, SELECT subquery, DATE_ADD expression, asterisk, EXTRACT expression, DATE, NULL, compound identifier, string concatenation expression, identifier, MySQL-style global variable, array element expression, TIMESTAMP operator expression, element of expression, expression with prefix unary operator, DATE operator expression, list, delimited by operator of variable arity, DATE_DIFF, parenthesized expression, COLUMNS matcher, RIGHT expression, array, TRIM, DATE_DIFF expression, case, string literal, unary minus expression, INTERVAL operator expression, DATESUB, TIMESTAMPSUB, CAST AS expression, INTERVAL, CASE, LEFT expression, LEFT, CAST operator, CAST, additive expression, SUBSTRING, list of elements, RTRIM, DATEDIFF, SUBSTRING expression, BETWEEN expression, function, DATEADD, DATE_SUB, tuple element expression, collection of literals, TIMESTAMP_ADD, list, delimited by binary operators, CAST expression, token, COLUMNS, TIMESTAMP_SUB, LTRIM, multiplicative expression, Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8b6cbba in /usr/bin/clickhouse | |
| 1. DB::parseQueryAndMovePosition(DB::IParser&, char const*&, char const*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, bool, unsigned long, unsigned long) @ 0x1234b208 in /usr/bin/clickhouse | |
| 2. ? @ 0xfc08f48 in /usr/bin/clickhouse | |
| 3. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xfc0dfd5 in /usr/bin/clickhouse | |
| 4. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0x103c36bf in /usr/bin/clickhouse | |
| 5. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x103c6e49 in /usr/bin/clickhouse | |
| 6. DB::HTTPServerConnection::run() @ 0x1044acd0 in /usr/bin/clickhouse | |
| 7. Poco::Net::TCPServerConnection::start() @ 0x12a3fd4f in /usr/bin/clickhouse | |
| 8. Poco::Net::TCPServerDispatcher::run() @ 0x12a417da in /usr/bin/clickhouse | |
| 9. Poco::PooledThread::run() @ 0x12b7ab39 in /usr/bin/clickhouse | |
| 10. Poco::ThreadImpl::runnableEntry(void*) @ 0x12b76b2a in /usr/bin/clickhouse | |
| 11. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 12. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.6.5.37 (official build)) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| 12. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.5.6.6 (official build)) | |
| 2021.06.24 11:23:01.072986 [ 9157 ] {822f6b8e-75a5-4f66-81b4-650ab0b4fa52} <Error> executeQuery: Code: 244, e.displayText() = DB::Exception: Unexpected logical error while adding block 347075 with ID '202106_10353628517386940799_6446266766293363453': No node, path /clickhouse/analytics/tables/01/events_local/replicas/02/parts/202106_347075_347075_0 (version 21.5.6.6 (official build)) (from 172.31.18.125:56032) (in query: INSERT INTO events (`id`, `event_type`, `date`, `customer_id`, `engine_id`, `query`, `display_width_px`, `display_height_px`, `user_agent`, `referrer_url`, `referrer_title`, `browser_language`, `ip_address`, `page_number`, `host_name`, `session_id`, `query_session_id`, `query_raw`, `device_type`, `geo_continent_name`, `geo_city_name`, `geo_country_iso_code`, `geo_region_name`, `geo_latitude`, `geo_longitude`, `click_link_url`, `click_link_index`, `click_clicked_object_id`, `click_title`, `click_query_id`, `click_context`, `query_type`, `query_total_banners`, `query_total_documents`, `query_response_time_ms`, `query_actual_query`, `query_quick_link_id`, `query_results_per_page`, `query_environment`, `query_request_type`, `query_banners`, `query_billable`, `application_name`, `application_version`, `http_status_code`, `aws_region`) FORMAT RowBinary ), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87a28ba in /usr/bin/clickhouse | |
| 1. DB::ReplicatedMergeTreeBlockOutputStream::commitPart(std::__1::shared_ptr<zkutil::ZooKeeper>&, std::__1::shared_ptr<DB::IMergeTreeDataPart>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0xfc17295 in /usr/bin/clickhouse | |
| 2. DB::ReplicatedMergeTreeBlockOutputStream::write(DB::Block const&) @ 0xfc133fd in /usr/bin/clickhouse | |
| 3. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 4. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 5. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 6. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0xfa175a2 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0xfa14109 in /usr/bin/clickhouse | |
| 9. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0xfa13334 in /usr/bin/clickhouse | |
| 10. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 11. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 13. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 14. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xeceefdd in /usr/bin/clickhouse | |
| 15. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf0a3879 in /usr/bin/clickhouse | |
| 16. DB::IBlockInputStream::read() @ 0xeccbbf5 in /usr/bin/clickhouse | |
| 17. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xecef236 in /usr/bin/clickhouse | |
| 18. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xf5ac165 in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0xfd2716f in /usr/bin/clickhouse | |
| 20. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0xfd2a889 in /usr/bin/clickhouse | |
| 21. DB::HTTPServerConnection::run() @ 0xfdae6f0 in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerConnection::start() @ 0x124232af in /usr/bin/clickhouse | |
| 23. Poco::Net::TCPServerDispatcher::run() @ 0x12424d3a in /usr/bin/clickhouse | |
| 24. Poco::PooledThread::run() @ 0x1255eaf9 in /usr/bin/clickhouse | |
| 25. Poco::ThreadImpl::runnableEntry(void*) @ 0x1255aaea in /usr/bin/clickhouse | |
| 26. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 27. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| 2021.06.24 11:23:01.073268 [ 9157 ] {822f6b8e-75a5-4f66-81b4-650ab0b4fa52} <Error> DynamicQueryHandler: Code: 244, e.displayText() = DB::Exception: Unexpected logical error while adding block 347075 with ID '202106_10353628517386940799_6446266766293363453': No node, path /clickhouse/analytics/tables/01/events_local/replicas/02/parts/202106_347075_347075_0, Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87a28ba in /usr/bin/clickhouse | |
| 1. DB::ReplicatedMergeTreeBlockOutputStream::commitPart(std::__1::shared_ptr<zkutil::ZooKeeper>&, std::__1::shared_ptr<DB::IMergeTreeDataPart>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0xfc17295 in /usr/bin/clickhouse | |
| 2. DB::ReplicatedMergeTreeBlockOutputStream::write(DB::Block const&) @ 0xfc133fd in /usr/bin/clickhouse | |
| 3. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 4. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 5. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 6. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0xfa175a2 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0xfa14109 in /usr/bin/clickhouse | |
| 9. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0xfa13334 in /usr/bin/clickhouse | |
| 10. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 11. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 13. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 14. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xeceefdd in /usr/bin/clickhouse | |
| 15. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf0a3879 in /usr/bin/clickhouse | |
| 16. DB::IBlockInputStream::read() @ 0xeccbbf5 in /usr/bin/clickhouse | |
| 17. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xecef236 in /usr/bin/clickhouse | |
| 18. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xf5ac165 in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0xfd2716f in /usr/bin/clickhouse | |
| 20. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0xfd2a889 in /usr/bin/clickhouse | |
| 21. DB::HTTPServerConnection::run() @ 0xfdae6f0 in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerConnection::start() @ 0x124232af in /usr/bin/clickhouse | |
| 23. Poco::Net::TCPServerDispatcher::run() @ 0x12424d3a in /usr/bin/clickhouse | |
| 24. Poco::PooledThread::run() @ 0x1255eaf9 in /usr/bin/clickhouse | |
| 25. Poco::ThreadImpl::runnableEntry(void*) @ 0x1255aaea in /usr/bin/clickhouse | |
| 26. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 27. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.5.6.6 (official build)) | |
| 2021.06.24 11:23:01.822658 [ 9157 ] {2223335b-39eb-4c68-ab37-90baf2a0498e} <Warning> ClusterProxy::SelectStreamFactory: Local replica of shard 1 is stale (delay: 397032s.) | |
| 2021.06.24 11:23:02.061966 [ 642 ] {} <Error> analytics.events_local_temp (89b20029-538f-434d-a1e3-e048226e872b): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 999, e.displayText() = Coordination::Exception: Can't get data for node /clickhouse/analytics/tables/01/events_local/replicas/02/log_pointer: node doesn't exist (No node), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87a28ba in /usr/bin/clickhouse | |
| 1. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error, int) @ 0x100ff595 in /usr/bin/clickhouse | |
| 2. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error) @ 0x100ff822 in /usr/bin/clickhouse | |
| 3. zkutil::ZooKeeper::get(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Stat*, std::__1::shared_ptr<Poco::Event> const&) @ 0x10106d12 in /usr/bin/clickhouse | |
| 4. DB::ReplicatedMergeTreeQueue::pullLogsToQueue(std::__1::shared_ptr<zkutil::ZooKeeper>, std::__1::function<void (Coordination::WatchResponse const&)>) @ 0xfc44167 in /usr/bin/clickhouse | |
| 5. DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0xf8945ca in /usr/bin/clickhouse | |
| 6. DB::BackgroundSchedulePoolTaskInfo::execute() @ 0xef0cc20 in /usr/bin/clickhouse | |
| 7. DB::BackgroundSchedulePool::threadFunction() @ 0xef0ec77 in /usr/bin/clickhouse | |
| 8. ? @ 0xef0f9f2 in /usr/bin/clickhouse | |
| 9. ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0x87e2b5f in /usr/bin/clickhouse | |
| 10. ? @ 0x87e6083 in /usr/bin/clickhouse | |
| 11. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 12. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.5.6.6 (official build)) | |
| 2021.06.24 11:23:02.232838 [ 9157 ] {9cf4ef0f-4669-4a45-9e23-a9a1b666f950} <Error> executeQuery: Code: 244, e.displayText() = DB::Exception: Unexpected logical error while adding block 347077 with ID '202106_10353628517386940799_6446266766293363453': No node, path /clickhouse/analytics/tables/01/events_local/replicas/02/parts/202106_347077_347077_0 (version 21.5.6.6 (official build)) (from 172.31.18.125:56044) (in query: INSERT INTO events (`id`, `event_type`, `date`, `customer_id`, `engine_id`, `query`, `display_width_px`, `display_height_px`, `user_agent`, `referrer_url`, `referrer_title`, `browser_language`, `ip_address`, `page_number`, `host_name`, `session_id`, `query_session_id`, `query_raw`, `device_type`, `geo_continent_name`, `geo_city_name`, `geo_country_iso_code`, `geo_region_name`, `geo_latitude`, `geo_longitude`, `click_link_url`, `click_link_index`, `click_clicked_object_id`, `click_title`, `click_query_id`, `click_context`, `query_type`, `query_total_banners`, `query_total_documents`, `query_response_time_ms`, `query_actual_query`, `query_quick_link_id`, `query_results_per_page`, `query_environment`, `query_request_type`, `query_banners`, `query_billable`, `application_name`, `application_version`, `http_status_code`, `aws_region`) FORMAT RowBinary ), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87a28ba in /usr/bin/clickhouse | |
| 1. DB::ReplicatedMergeTreeBlockOutputStream::commitPart(std::__1::shared_ptr<zkutil::ZooKeeper>&, std::__1::shared_ptr<DB::IMergeTreeDataPart>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0xfc17295 in /usr/bin/clickhouse | |
| 2. DB::ReplicatedMergeTreeBlockOutputStream::write(DB::Block const&) @ 0xfc133fd in /usr/bin/clickhouse | |
| 3. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 4. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 5. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 6. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0xfa175a2 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0xfa14109 in /usr/bin/clickhouse | |
| 9. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0xfa13334 in /usr/bin/clickhouse | |
| 10. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 11. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 13. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 14. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xeceefdd in /usr/bin/clickhouse | |
| 15. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf0a3879 in /usr/bin/clickhouse | |
| 16. DB::IBlockInputStream::read() @ 0xeccbbf5 in /usr/bin/clickhouse | |
| 17. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xecef236 in /usr/bin/clickhouse | |
| 18. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xf5ac165 in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0xfd2716f in /usr/bin/clickhouse | |
| 20. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0xfd2a889 in /usr/bin/clickhouse | |
| 21. DB::HTTPServerConnection::run() @ 0xfdae6f0 in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerConnection::start() @ 0x124232af in /usr/bin/clickhouse | |
| 23. Poco::Net::TCPServerDispatcher::run() @ 0x12424d3a in /usr/bin/clickhouse | |
| 24. Poco::PooledThread::run() @ 0x1255eaf9 in /usr/bin/clickhouse | |
| 25. Poco::ThreadImpl::runnableEntry(void*) @ 0x1255aaea in /usr/bin/clickhouse | |
| 26. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 27. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| 2021.06.24 11:23:02.233112 [ 9157 ] {9cf4ef0f-4669-4a45-9e23-a9a1b666f950} <Error> DynamicQueryHandler: Code: 244, e.displayText() = DB::Exception: Unexpected logical error while adding block 347077 with ID '202106_10353628517386940799_6446266766293363453': No node, path /clickhouse/analytics/tables/01/events_local/replicas/02/parts/202106_347077_347077_0, Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87a28ba in /usr/bin/clickhouse | |
| 1. DB::ReplicatedMergeTreeBlockOutputStream::commitPart(std::__1::shared_ptr<zkutil::ZooKeeper>&, std::__1::shared_ptr<DB::IMergeTreeDataPart>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0xfc17295 in /usr/bin/clickhouse | |
| 2. DB::ReplicatedMergeTreeBlockOutputStream::write(DB::Block const&) @ 0xfc133fd in /usr/bin/clickhouse | |
| 3. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 4. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 5. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 6. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0xfa175a2 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0xfa14109 in /usr/bin/clickhouse | |
| 9. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0xfa13334 in /usr/bin/clickhouse | |
| 10. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 11. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 13. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 14. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xeceefdd in /usr/bin/clickhouse | |
| 15. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf0a3879 in /usr/bin/clickhouse | |
| 16. DB::IBlockInputStream::read() @ 0xeccbbf5 in /usr/bin/clickhouse | |
| 17. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xecef236 in /usr/bin/clickhouse | |
| 18. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xf5ac165 in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0xfd2716f in /usr/bin/clickhouse | |
| 20. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0xfd2a889 in /usr/bin/clickhouse | |
| 21. DB::HTTPServerConnection::run() @ 0xfdae6f0 in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerConnection::start() @ 0x124232af in /usr/bin/clickhouse | |
| 23. Poco::Net::TCPServerDispatcher::run() @ 0x12424d3a in /usr/bin/clickhouse | |
| 24. Poco::PooledThread::run() @ 0x1255eaf9 in /usr/bin/clickhouse | |
| 25. Poco::ThreadImpl::runnableEntry(void*) @ 0x1255aaea in /usr/bin/clickhouse | |
| 26. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 27. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.5.6.6 (official build)) | |
| 2021.06.24 11:23:03.062801 [ 639 ] {} <Error> analytics.events_local_temp (89b20029-538f-434d-a1e3-e048226e872b): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 999, e.displayText() = Coordination::Exception: Can't get data for node /clickhouse/analytics/tables/01/events_local/replicas/02/log_pointer: node doesn't exist (No node), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87a28ba in /usr/bin/clickhouse | |
| 1. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error, int) @ 0x100ff595 in /usr/bin/clickhouse | |
| 2. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error) @ 0x100ff822 in /usr/bin/clickhouse | |
| 3. zkutil::ZooKeeper::get(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Stat*, std::__1::shared_ptr<Poco::Event> const&) @ 0x10106d12 in /usr/bin/clickhouse | |
| 4. DB::ReplicatedMergeTreeQueue::pullLogsToQueue(std::__1::shared_ptr<zkutil::ZooKeeper>, std::__1::function<void (Coordination::WatchResponse const&)>) @ 0xfc44167 in /usr/bin/clickhouse | |
| 5. DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0xf8945ca in /usr/bin/clickhouse | |
| 6. DB::BackgroundSchedulePoolTaskInfo::execute() @ 0xef0cc20 in /usr/bin/clickhouse | |
| 7. DB::BackgroundSchedulePool::threadFunction() @ 0xef0ec77 in /usr/bin/clickhouse | |
| 8. ? @ 0xef0f9f2 in /usr/bin/clickhouse | |
| 9. ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0x87e2b5f in /usr/bin/clickhouse | |
| 10. ? @ 0x87e6083 in /usr/bin/clickhouse | |
| 11. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 12. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.5.6.6 (official build)) | |
| 2021.06.24 11:23:03.265647 [ 9157 ] {5520ff43-411c-4e9e-a426-c6e7c7e29d74} <Warning> ClusterProxy::SelectStreamFactory: Local replica of shard 1 is stale (delay: 397034s.) | |
| 2021.06.24 11:23:03.387460 [ 9157 ] {dca2e546-3861-4c6d-925d-80aa4d0c30a7} <Error> executeQuery: Code: 244, e.displayText() = DB::Exception: Unexpected logical error while adding block 347079 with ID '202106_14545680890394358858_9100061647636589060': No node, path /clickhouse/analytics/tables/01/events_local/replicas/02/parts/202106_347079_347079_0 (version 21.5.6.6 (official build)) (from 172.31.18.125:56056) (in query: INSERT INTO events (`id`, `event_type`, `date`, `customer_id`, `engine_id`, `query`, `display_width_px`, `display_height_px`, `user_agent`, `referrer_url`, `referrer_title`, `browser_language`, `ip_address`, `page_number`, `host_name`, `session_id`, `query_session_id`, `query_raw`, `device_type`, `geo_continent_name`, `geo_city_name`, `geo_country_iso_code`, `geo_region_name`, `geo_latitude`, `geo_longitude`, `click_link_url`, `click_link_index`, `click_clicked_object_id`, `click_title`, `click_query_id`, `click_context`, `query_type`, `query_total_banners`, `query_total_documents`, `query_response_time_ms`, `query_actual_query`, `query_quick_link_id`, `query_results_per_page`, `query_environment`, `query_request_type`, `query_banners`, `query_billable`, `application_name`, `application_version`, `http_status_code`, `aws_region`) FORMAT RowBinary ), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87a28ba in /usr/bin/clickhouse | |
| 1. DB::ReplicatedMergeTreeBlockOutputStream::commitPart(std::__1::shared_ptr<zkutil::ZooKeeper>&, std::__1::shared_ptr<DB::IMergeTreeDataPart>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0xfc17295 in /usr/bin/clickhouse | |
| 2. DB::ReplicatedMergeTreeBlockOutputStream::write(DB::Block const&) @ 0xfc133fd in /usr/bin/clickhouse | |
| 3. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 4. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 5. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 6. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0xfa175a2 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0xfa14109 in /usr/bin/clickhouse | |
| 9. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0xfa13334 in /usr/bin/clickhouse | |
| 10. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 11. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 13. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 14. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xeceefdd in /usr/bin/clickhouse | |
| 15. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf0a3879 in /usr/bin/clickhouse | |
| 16. DB::IBlockInputStream::read() @ 0xeccbbf5 in /usr/bin/clickhouse | |
| 17. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xecef236 in /usr/bin/clickhouse | |
| 18. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xf5ac165 in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0xfd2716f in /usr/bin/clickhouse | |
| 20. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0xfd2a889 in /usr/bin/clickhouse | |
| 21. DB::HTTPServerConnection::run() @ 0xfdae6f0 in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerConnection::start() @ 0x124232af in /usr/bin/clickhouse | |
| 23. Poco::Net::TCPServerDispatcher::run() @ 0x12424d3a in /usr/bin/clickhouse | |
| 24. Poco::PooledThread::run() @ 0x1255eaf9 in /usr/bin/clickhouse | |
| 25. Poco::ThreadImpl::runnableEntry(void*) @ 0x1255aaea in /usr/bin/clickhouse | |
| 26. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 27. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| 2021.06.24 11:23:03.387714 [ 9157 ] {dca2e546-3861-4c6d-925d-80aa4d0c30a7} <Error> DynamicQueryHandler: Code: 244, e.displayText() = DB::Exception: Unexpected logical error while adding block 347079 with ID '202106_14545680890394358858_9100061647636589060': No node, path /clickhouse/analytics/tables/01/events_local/replicas/02/parts/202106_347079_347079_0, Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87a28ba in /usr/bin/clickhouse | |
| 1. DB::ReplicatedMergeTreeBlockOutputStream::commitPart(std::__1::shared_ptr<zkutil::ZooKeeper>&, std::__1::shared_ptr<DB::IMergeTreeDataPart>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0xfc17295 in /usr/bin/clickhouse | |
| 2. DB::ReplicatedMergeTreeBlockOutputStream::write(DB::Block const&) @ 0xfc133fd in /usr/bin/clickhouse | |
| 3. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 4. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 5. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 6. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0xfa175a2 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0xfa14109 in /usr/bin/clickhouse | |
| 9. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0xfa13334 in /usr/bin/clickhouse | |
| 10. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 11. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 13. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 14. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xeceefdd in /usr/bin/clickhouse | |
| 15. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf0a3879 in /usr/bin/clickhouse | |
| 16. DB::IBlockInputStream::read() @ 0xeccbbf5 in /usr/bin/clickhouse | |
| 17. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xecef236 in /usr/bin/clickhouse | |
| 18. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xf5ac165 in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0xfd2716f in /usr/bin/clickhouse | |
| 20. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0xfd2a889 in /usr/bin/clickhouse | |
| 21. DB::HTTPServerConnection::run() @ 0xfdae6f0 in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerConnection::start() @ 0x124232af in /usr/bin/clickhouse | |
| 23. Poco::Net::TCPServerDispatcher::run() @ 0x12424d3a in /usr/bin/clickhouse | |
| 24. Poco::PooledThread::run() @ 0x1255eaf9 in /usr/bin/clickhouse | |
| 25. Poco::ThreadImpl::runnableEntry(void*) @ 0x1255aaea in /usr/bin/clickhouse | |
| 26. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 27. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.5.6.6 (official build)) | |
| 2021.06.24 11:23:03.839517 [ 650 ] {} <Error> analytics.events_local_temp (89b20029-538f-434d-a1e3-e048226e872b): void DB::StorageReplicatedMergeTree::mergeSelectingTask(): Code: 999, e.displayText() = Coordination::Exception: Can't get data for node /clickhouse/analytics/tables/01/events_local/replicas/02/log_pointer: node doesn't exist (No node), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87a28ba in /usr/bin/clickhouse | |
| 1. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error, int) @ 0x100ff595 in /usr/bin/clickhouse | |
| 2. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error) @ 0x100ff822 in /usr/bin/clickhouse | |
| 3. zkutil::ZooKeeper::get(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Stat*, std::__1::shared_ptr<Poco::Event> const&) @ 0x10106d12 in /usr/bin/clickhouse | |
| 4. DB::ReplicatedMergeTreeQueue::pullLogsToQueue(std::__1::shared_ptr<zkutil::ZooKeeper>, std::__1::function<void (Coordination::WatchResponse const&)>) @ 0xfc44167 in /usr/bin/clickhouse | |
| 5. DB::ReplicatedMergeTreeMergePredicate::ReplicatedMergeTreeMergePredicate(DB::ReplicatedMergeTreeQueue&, std::__1::shared_ptr<zkutil::ZooKeeper>&) @ 0xfc60e15 in /usr/bin/clickhouse | |
| 6. DB::StorageReplicatedMergeTree::mergeSelectingTask() @ 0xf895b4a in /usr/bin/clickhouse | |
| 7. DB::BackgroundSchedulePoolTaskInfo::execute() @ 0xef0cc20 in /usr/bin/clickhouse | |
| 8. DB::BackgroundSchedulePool::threadFunction() @ 0xef0ec77 in /usr/bin/clickhouse | |
| 9. ? @ 0xef0f9f2 in /usr/bin/clickhouse | |
| 10. ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0x87e2b5f in /usr/bin/clickhouse | |
| 11. ? @ 0x87e6083 in /usr/bin/clickhouse | |
| 12. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 13. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.5.6.6 (official build)) | |
| 2021.06.24 11:23:03.872339 [ 9157 ] {42159554-6ef9-4c87-aecf-d51534eca2b8} <Warning> ClusterProxy::SelectStreamFactory: Local replica of shard 1 is stale (delay: 397034s.) | |
| 2021.06.24 11:23:04.063719 [ 647 ] {} <Error> analytics.events_local_temp (89b20029-538f-434d-a1e3-e048226e872b): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 999, e.displayText() = Coordination::Exception: Can't get data for node /clickhouse/analytics/tables/01/events_local/replicas/02/log_pointer: node doesn't exist (No node), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87a28ba in /usr/bin/clickhouse | |
| 1. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error, int) @ 0x100ff595 in /usr/bin/clickhouse | |
| 2. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error) @ 0x100ff822 in /usr/bin/clickhouse | |
| 3. zkutil::ZooKeeper::get(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Stat*, std::__1::shared_ptr<Poco::Event> const&) @ 0x10106d12 in /usr/bin/clickhouse | |
| 4. DB::ReplicatedMergeTreeQueue::pullLogsToQueue(std::__1::shared_ptr<zkutil::ZooKeeper>, std::__1::function<void (Coordination::WatchResponse const&)>) @ 0xfc44167 in /usr/bin/clickhouse | |
| 5. DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0xf8945ca in /usr/bin/clickhouse | |
| 6. DB::BackgroundSchedulePoolTaskInfo::execute() @ 0xef0cc20 in /usr/bin/clickhouse | |
| 7. DB::BackgroundSchedulePool::threadFunction() @ 0xef0ec77 in /usr/bin/clickhouse | |
| 8. ? @ 0xef0f9f2 in /usr/bin/clickhouse | |
| 9. ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0x87e2b5f in /usr/bin/clickhouse | |
| 10. ? @ 0x87e6083 in /usr/bin/clickhouse | |
| 11. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 12. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.5.6.6 (official build)) | |
| 2021.06.24 11:23:04.457610 [ 9157 ] {803f2e57-ac80-4c0b-b357-06713fda95ac} <Error> executeQuery: Code: 244, e.displayText() = DB::Exception: Unexpected logical error while adding block 347081 with ID '202106_14545680890394358858_9100061647636589060': No node, path /clickhouse/analytics/tables/01/events_local/replicas/02/parts/202106_347081_347081_0 (version 21.5.6.6 (official build)) (from 172.31.18.125:56068) (in query: INSERT INTO events (`id`, `event_type`, `date`, `customer_id`, `engine_id`, `query`, `display_width_px`, `display_height_px`, `user_agent`, `referrer_url`, `referrer_title`, `browser_language`, `ip_address`, `page_number`, `host_name`, `session_id`, `query_session_id`, `query_raw`, `device_type`, `geo_continent_name`, `geo_city_name`, `geo_country_iso_code`, `geo_region_name`, `geo_latitude`, `geo_longitude`, `click_link_url`, `click_link_index`, `click_clicked_object_id`, `click_title`, `click_query_id`, `click_context`, `query_type`, `query_total_banners`, `query_total_documents`, `query_response_time_ms`, `query_actual_query`, `query_quick_link_id`, `query_results_per_page`, `query_environment`, `query_request_type`, `query_banners`, `query_billable`, `application_name`, `application_version`, `http_status_code`, `aws_region`) FORMAT RowBinary ), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87a28ba in /usr/bin/clickhouse | |
| 1. DB::ReplicatedMergeTreeBlockOutputStream::commitPart(std::__1::shared_ptr<zkutil::ZooKeeper>&, std::__1::shared_ptr<DB::IMergeTreeDataPart>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0xfc17295 in /usr/bin/clickhouse | |
| 2. DB::ReplicatedMergeTreeBlockOutputStream::write(DB::Block const&) @ 0xfc133fd in /usr/bin/clickhouse | |
| 3. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 4. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 5. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 6. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0xfa175a2 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0xfa14109 in /usr/bin/clickhouse | |
| 9. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0xfa13334 in /usr/bin/clickhouse | |
| 10. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 11. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 13. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 14. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xeceefdd in /usr/bin/clickhouse | |
| 15. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf0a3879 in /usr/bin/clickhouse | |
| 16. DB::IBlockInputStream::read() @ 0xeccbbf5 in /usr/bin/clickhouse | |
| 17. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xecef236 in /usr/bin/clickhouse | |
| 18. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xf5ac165 in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0xfd2716f in /usr/bin/clickhouse | |
| 20. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0xfd2a889 in /usr/bin/clickhouse | |
| 21. DB::HTTPServerConnection::run() @ 0xfdae6f0 in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerConnection::start() @ 0x124232af in /usr/bin/clickhouse | |
| 23. Poco::Net::TCPServerDispatcher::run() @ 0x12424d3a in /usr/bin/clickhouse | |
| 24. Poco::PooledThread::run() @ 0x1255eaf9 in /usr/bin/clickhouse | |
| 25. Poco::ThreadImpl::runnableEntry(void*) @ 0x1255aaea in /usr/bin/clickhouse | |
| 26. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 27. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| 2021.06.24 11:23:04.457855 [ 9157 ] {803f2e57-ac80-4c0b-b357-06713fda95ac} <Error> DynamicQueryHandler: Code: 244, e.displayText() = DB::Exception: Unexpected logical error while adding block 347081 with ID '202106_14545680890394358858_9100061647636589060': No node, path /clickhouse/analytics/tables/01/events_local/replicas/02/parts/202106_347081_347081_0, Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87a28ba in /usr/bin/clickhouse | |
| 1. DB::ReplicatedMergeTreeBlockOutputStream::commitPart(std::__1::shared_ptr<zkutil::ZooKeeper>&, std::__1::shared_ptr<DB::IMergeTreeDataPart>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0xfc17295 in /usr/bin/clickhouse | |
| 2. DB::ReplicatedMergeTreeBlockOutputStream::write(DB::Block const&) @ 0xfc133fd in /usr/bin/clickhouse | |
| 3. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 4. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 5. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 6. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0xfa175a2 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0xfa14109 in /usr/bin/clickhouse | |
| 9. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0xfa13334 in /usr/bin/clickhouse | |
| 10. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 11. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 13. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 14. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xeceefdd in /usr/bin/clickhouse | |
| 15. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf0a3879 in /usr/bin/clickhouse | |
| 16. DB::IBlockInputStream::read() @ 0xeccbbf5 in /usr/bin/clickhouse | |
| 17. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xecef236 in /usr/bin/clickhouse | |
| 18. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xf5ac165 in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0xfd2716f in /usr/bin/clickhouse | |
| 20. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0xfd2a889 in /usr/bin/clickhouse | |
| 21. DB::HTTPServerConnection::run() @ 0xfdae6f0 in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerConnection::start() @ 0x124232af in /usr/bin/clickhouse | |
| 23. Poco::Net::TCPServerDispatcher::run() @ 0x12424d3a in /usr/bin/clickhouse | |
| 24. Poco::PooledThread::run() @ 0x1255eaf9 in /usr/bin/clickhouse | |
| 25. Poco::ThreadImpl::runnableEntry(void*) @ 0x1255aaea in /usr/bin/clickhouse | |
| 26. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 27. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.5.6.6 (official build)) | |
| 2021.06.24 11:23:05.064615 [ 637 ] {} <Error> analytics.events_local_temp (89b20029-538f-434d-a1e3-e048226e872b): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 999, e.displayText() = Coordination::Exception: Can't get data for node /clickhouse/analytics/tables/01/events_local/replicas/02/log_pointer: node doesn't exist (No node), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87a28ba in /usr/bin/clickhouse | |
| 1. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error, int) @ 0x100ff595 in /usr/bin/clickhouse | |
| 2. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error) @ 0x100ff822 in /usr/bin/clickhouse | |
| 3. zkutil::ZooKeeper::get(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Stat*, std::__1::shared_ptr<Poco::Event> const&) @ 0x10106d12 in /usr/bin/clickhouse | |
| 4. DB::ReplicatedMergeTreeQueue::pullLogsToQueue(std::__1::shared_ptr<zkutil::ZooKeeper>, std::__1::function<void (Coordination::WatchResponse const&)>) @ 0xfc44167 in /usr/bin/clickhouse | |
| 5. DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0xf8945ca in /usr/bin/clickhouse | |
| 6. DB::BackgroundSchedulePoolTaskInfo::execute() @ 0xef0cc20 in /usr/bin/clickhouse | |
| 7. DB::BackgroundSchedulePool::threadFunction() @ 0xef0ec77 in /usr/bin/clickhouse | |
| 8. ? @ 0xef0f9f2 in /usr/bin/clickhouse | |
| 9. ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0x87e2b5f in /usr/bin/clickhouse | |
| 10. ? @ 0x87e6083 in /usr/bin/clickhouse | |
| 11. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 12. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.5.6.6 (official build)) | |
| 2021.06.24 11:23:05.463554 [ 9157 ] {6ab1bf27-0e12-4454-b57f-174c1bfa83b3} <Warning> ClusterProxy::SelectStreamFactory: Local replica of shard 1 is stale (delay: 397036s.) | |
| 2021.06.24 11:23:05.586408 [ 9157 ] {4f61143f-06bf-483a-bc57-f5a5683be36b} <Error> executeQuery: Code: 244, e.displayText() = DB::Exception: Unexpected logical error while adding block 347083 with ID '202106_15671413042809523583_14831841158419873903': No node, path /clickhouse/analytics/tables/01/events_local/replicas/02/parts/202106_347083_347083_0 (version 21.5.6.6 (official build)) (from 172.31.18.125:56080) (in query: INSERT INTO events (`id`, `event_type`, `date`, `customer_id`, `engine_id`, `query`, `display_width_px`, `display_height_px`, `user_agent`, `referrer_url`, `referrer_title`, `browser_language`, `ip_address`, `page_number`, `host_name`, `session_id`, `query_session_id`, `query_raw`, `device_type`, `geo_continent_name`, `geo_city_name`, `geo_country_iso_code`, `geo_region_name`, `geo_latitude`, `geo_longitude`, `click_link_url`, `click_link_index`, `click_clicked_object_id`, `click_title`, `click_query_id`, `click_context`, `query_type`, `query_total_banners`, `query_total_documents`, `query_response_time_ms`, `query_actual_query`, `query_quick_link_id`, `query_results_per_page`, `query_environment`, `query_request_type`, `query_banners`, `query_billable`, `application_name`, `application_version`, `http_status_code`, `aws_region`) FORMAT RowBinary ), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87a28ba in /usr/bin/clickhouse | |
| 1. DB::ReplicatedMergeTreeBlockOutputStream::commitPart(std::__1::shared_ptr<zkutil::ZooKeeper>&, std::__1::shared_ptr<DB::IMergeTreeDataPart>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0xfc17295 in /usr/bin/clickhouse | |
| 2. DB::ReplicatedMergeTreeBlockOutputStream::write(DB::Block const&) @ 0xfc133fd in /usr/bin/clickhouse | |
| 3. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 4. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 5. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 6. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0xfa175a2 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0xfa14109 in /usr/bin/clickhouse | |
| 9. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0xfa13334 in /usr/bin/clickhouse | |
| 10. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 11. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 13. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 14. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xeceefdd in /usr/bin/clickhouse | |
| 15. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf0a3879 in /usr/bin/clickhouse | |
| 16. DB::IBlockInputStream::read() @ 0xeccbbf5 in /usr/bin/clickhouse | |
| 17. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xecef236 in /usr/bin/clickhouse | |
| 18. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xf5ac165 in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0xfd2716f in /usr/bin/clickhouse | |
| 20. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0xfd2a889 in /usr/bin/clickhouse | |
| 21. DB::HTTPServerConnection::run() @ 0xfdae6f0 in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerConnection::start() @ 0x124232af in /usr/bin/clickhouse | |
| 23. Poco::Net::TCPServerDispatcher::run() @ 0x12424d3a in /usr/bin/clickhouse | |
| 24. Poco::PooledThread::run() @ 0x1255eaf9 in /usr/bin/clickhouse | |
| 25. Poco::ThreadImpl::runnableEntry(void*) @ 0x1255aaea in /usr/bin/clickhouse | |
| 26. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 27. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| 2021.06.24 11:23:05.586683 [ 9157 ] {4f61143f-06bf-483a-bc57-f5a5683be36b} <Error> DynamicQueryHandler: Code: 244, e.displayText() = DB::Exception: Unexpected logical error while adding block 347083 with ID '202106_15671413042809523583_14831841158419873903': No node, path /clickhouse/analytics/tables/01/events_local/replicas/02/parts/202106_347083_347083_0, Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87a28ba in /usr/bin/clickhouse | |
| 1. DB::ReplicatedMergeTreeBlockOutputStream::commitPart(std::__1::shared_ptr<zkutil::ZooKeeper>&, std::__1::shared_ptr<DB::IMergeTreeDataPart>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0xfc17295 in /usr/bin/clickhouse | |
| 2. DB::ReplicatedMergeTreeBlockOutputStream::write(DB::Block const&) @ 0xfc133fd in /usr/bin/clickhouse | |
| 3. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 4. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 5. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 6. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0xfa175a2 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0xfa14109 in /usr/bin/clickhouse | |
| 9. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0xfa13334 in /usr/bin/clickhouse | |
| 10. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 11. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 13. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 14. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xeceefdd in /usr/bin/clickhouse | |
| 15. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf0a3879 in /usr/bin/clickhouse | |
| 16. DB::IBlockInputStream::read() @ 0xeccbbf5 in /usr/bin/clickhouse | |
| 17. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xecef236 in /usr/bin/clickhouse | |
| 18. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xf5ac165 in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0xfd2716f in /usr/bin/clickhouse | |
| 20. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0xfd2a889 in /usr/bin/clickhouse | |
| 21. DB::HTTPServerConnection::run() @ 0xfdae6f0 in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerConnection::start() @ 0x124232af in /usr/bin/clickhouse | |
| 23. Poco::Net::TCPServerDispatcher::run() @ 0x12424d3a in /usr/bin/clickhouse | |
| 24. Poco::PooledThread::run() @ 0x1255eaf9 in /usr/bin/clickhouse | |
| 25. Poco::ThreadImpl::runnableEntry(void*) @ 0x1255aaea in /usr/bin/clickhouse | |
| 26. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 27. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.5.6.6 (official build)) | |
| 2021.06.24 11:23:05.852381 [ 9157 ] {a95769cd-2387-48aa-b73f-a7c12a4e0c68} <Warning> ClusterProxy::SelectStreamFactory: Local replica of shard 1 is stale (delay: 397036s.) | |
| 2021.06.24 11:23:06.065443 [ 642 ] {} <Error> analytics.events_local_temp (89b20029-538f-434d-a1e3-e048226e872b): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 999, e.displayText() = Coordination::Exception: Can't get data for node /clickhouse/analytics/tables/01/events_local/replicas/02/log_pointer: node doesn't exist (No node), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87a28ba in /usr/bin/clickhouse | |
| 1. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error, int) @ 0x100ff595 in /usr/bin/clickhouse | |
| 2. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error) @ 0x100ff822 in /usr/bin/clickhouse | |
| 3. zkutil::ZooKeeper::get(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Stat*, std::__1::shared_ptr<Poco::Event> const&) @ 0x10106d12 in /usr/bin/clickhouse | |
| 4. DB::ReplicatedMergeTreeQueue::pullLogsToQueue(std::__1::shared_ptr<zkutil::ZooKeeper>, std::__1::function<void (Coordination::WatchResponse const&)>) @ 0xfc44167 in /usr/bin/clickhouse | |
| 5. DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0xf8945ca in /usr/bin/clickhouse | |
| 6. DB::BackgroundSchedulePoolTaskInfo::execute() @ 0xef0cc20 in /usr/bin/clickhouse | |
| 7. DB::BackgroundSchedulePool::threadFunction() @ 0xef0ec77 in /usr/bin/clickhouse | |
| 8. ? @ 0xef0f9f2 in /usr/bin/clickhouse | |
| 9. ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0x87e2b5f in /usr/bin/clickhouse | |
| 10. ? @ 0x87e6083 in /usr/bin/clickhouse | |
| 11. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 12. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.5.6.6 (official build)) | |
| 2021.06.24 11:23:06.799784 [ 9157 ] {60a6d121-5318-4c86-8254-03d94a227fee} <Error> executeQuery: Code: 244, e.displayText() = DB::Exception: Unexpected logical error while adding block 347085 with ID '202106_15671413042809523583_14831841158419873903': No node, path /clickhouse/analytics/tables/01/events_local/replicas/02/parts/202106_347085_347085_0 (version 21.5.6.6 (official build)) (from 172.31.18.125:56092) (in query: INSERT INTO events (`id`, `event_type`, `date`, `customer_id`, `engine_id`, `query`, `display_width_px`, `display_height_px`, `user_agent`, `referrer_url`, `referrer_title`, `browser_language`, `ip_address`, `page_number`, `host_name`, `session_id`, `query_session_id`, `query_raw`, `device_type`, `geo_continent_name`, `geo_city_name`, `geo_country_iso_code`, `geo_region_name`, `geo_latitude`, `geo_longitude`, `click_link_url`, `click_link_index`, `click_clicked_object_id`, `click_title`, `click_query_id`, `click_context`, `query_type`, `query_total_banners`, `query_total_documents`, `query_response_time_ms`, `query_actual_query`, `query_quick_link_id`, `query_results_per_page`, `query_environment`, `query_request_type`, `query_banners`, `query_billable`, `application_name`, `application_version`, `http_status_code`, `aws_region`) FORMAT RowBinary ), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87a28ba in /usr/bin/clickhouse | |
| 1. DB::ReplicatedMergeTreeBlockOutputStream::commitPart(std::__1::shared_ptr<zkutil::ZooKeeper>&, std::__1::shared_ptr<DB::IMergeTreeDataPart>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0xfc17295 in /usr/bin/clickhouse | |
| 2. DB::ReplicatedMergeTreeBlockOutputStream::write(DB::Block const&) @ 0xfc133fd in /usr/bin/clickhouse | |
| 3. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 4. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 5. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 6. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0xfa175a2 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0xfa14109 in /usr/bin/clickhouse | |
| 9. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0xfa13334 in /usr/bin/clickhouse | |
| 10. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 11. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 13. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 14. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xeceefdd in /usr/bin/clickhouse | |
| 15. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf0a3879 in /usr/bin/clickhouse | |
| 16. DB::IBlockInputStream::read() @ 0xeccbbf5 in /usr/bin/clickhouse | |
| 17. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xecef236 in /usr/bin/clickhouse | |
| 18. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xf5ac165 in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0xfd2716f in /usr/bin/clickhouse | |
| 20. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0xfd2a889 in /usr/bin/clickhouse | |
| 21. DB::HTTPServerConnection::run() @ 0xfdae6f0 in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerConnection::start() @ 0x124232af in /usr/bin/clickhouse | |
| 23. Poco::Net::TCPServerDispatcher::run() @ 0x12424d3a in /usr/bin/clickhouse | |
| 24. Poco::PooledThread::run() @ 0x1255eaf9 in /usr/bin/clickhouse | |
| 25. Poco::ThreadImpl::runnableEntry(void*) @ 0x1255aaea in /usr/bin/clickhouse | |
| 26. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 27. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| 2021.06.24 11:23:06.800074 [ 9157 ] {60a6d121-5318-4c86-8254-03d94a227fee} <Error> DynamicQueryHandler: Code: 244, e.displayText() = DB::Exception: Unexpected logical error while adding block 347085 with ID '202106_15671413042809523583_14831841158419873903': No node, path /clickhouse/analytics/tables/01/events_local/replicas/02/parts/202106_347085_347085_0, Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87a28ba in /usr/bin/clickhouse | |
| 1. DB::ReplicatedMergeTreeBlockOutputStream::commitPart(std::__1::shared_ptr<zkutil::ZooKeeper>&, std::__1::shared_ptr<DB::IMergeTreeDataPart>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0xfc17295 in /usr/bin/clickhouse | |
| 2. DB::ReplicatedMergeTreeBlockOutputStream::write(DB::Block const&) @ 0xfc133fd in /usr/bin/clickhouse | |
| 3. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 4. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 5. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 6. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0xfa175a2 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0xfa14109 in /usr/bin/clickhouse | |
| 9. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0xfa13334 in /usr/bin/clickhouse | |
| 10. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 11. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 13. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 14. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xeceefdd in /usr/bin/clickhouse | |
| 15. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf0a3879 in /usr/bin/clickhouse | |
| 16. DB::IBlockInputStream::read() @ 0xeccbbf5 in /usr/bin/clickhouse | |
| 17. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xecef236 in /usr/bin/clickhouse | |
| 18. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xf5ac165 in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0xfd2716f in /usr/bin/clickhouse | |
| 20. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0xfd2a889 in /usr/bin/clickhouse | |
| 21. DB::HTTPServerConnection::run() @ 0xfdae6f0 in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerConnection::start() @ 0x124232af in /usr/bin/clickhouse | |
| 23. Poco::Net::TCPServerDispatcher::run() @ 0x12424d3a in /usr/bin/clickhouse | |
| 24. Poco::PooledThread::run() @ 0x1255eaf9 in /usr/bin/clickhouse | |
| 25. Poco::ThreadImpl::runnableEntry(void*) @ 0x1255aaea in /usr/bin/clickhouse | |
| 26. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 27. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.5.6.6 (official build)) | |
| 2021.06.24 11:23:07.066468 [ 645 ] {} <Error> analytics.events_local_temp (89b20029-538f-434d-a1e3-e048226e872b): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 999, e.displayText() = Coordination::Exception: Can't get data for node /clickhouse/analytics/tables/01/events_local/replicas/02/log_pointer: node doesn't exist (No node), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87a28ba in /usr/bin/clickhouse | |
| 1. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error, int) @ 0x100ff595 in /usr/bin/clickhouse | |
| 2. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error) @ 0x100ff822 in /usr/bin/clickhouse | |
| 3. zkutil::ZooKeeper::get(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Stat*, std::__1::shared_ptr<Poco::Event> const&) @ 0x10106d12 in /usr/bin/clickhouse | |
| 4. DB::ReplicatedMergeTreeQueue::pullLogsToQueue(std::__1::shared_ptr<zkutil::ZooKeeper>, std::__1::function<void (Coordination::WatchResponse const&)>) @ 0xfc44167 in /usr/bin/clickhouse | |
| 5. DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0xf8945ca in /usr/bin/clickhouse | |
| 6. DB::BackgroundSchedulePoolTaskInfo::execute() @ 0xef0cc20 in /usr/bin/clickhouse | |
| 7. DB::BackgroundSchedulePool::threadFunction() @ 0xef0ec77 in /usr/bin/clickhouse | |
| 8. ? @ 0xef0f9f2 in /usr/bin/clickhouse | |
| 9. ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0x87e2b5f in /usr/bin/clickhouse | |
| 10. ? @ 0x87e6083 in /usr/bin/clickhouse | |
| 11. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 12. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.5.6.6 (official build)) | |
| 2021.06.24 11:23:07.807196 [ 9157 ] {810e488a-7fcb-42af-ad1c-6111f412d4c3} <Warning> ClusterProxy::SelectStreamFactory: Local replica of shard 1 is stale (delay: 397038s.) | |
| 2021.06.24 11:23:07.890169 [ 9157 ] {1e8b71f3-3460-4b8d-83c8-05f8b45b2efd} <Warning> ClusterProxy::SelectStreamFactory: Local replica of shard 1 is stale (delay: 397038s.) | |
| 2021.06.24 11:23:07.963626 [ 9157 ] {462c4cfb-0fff-49d4-aed7-bafb08c68e6b} <Error> executeQuery: Code: 244, e.displayText() = DB::Exception: Unexpected logical error while adding block 347087 with ID '202106_8149276849507687568_17514355474010194501': No node, path /clickhouse/analytics/tables/01/events_local/replicas/02/parts/202106_347087_347087_0 (version 21.5.6.6 (official build)) (from 172.31.18.125:56108) (in query: INSERT INTO events (`id`, `event_type`, `date`, `customer_id`, `engine_id`, `query`, `display_width_px`, `display_height_px`, `user_agent`, `referrer_url`, `referrer_title`, `browser_language`, `ip_address`, `page_number`, `host_name`, `session_id`, `query_session_id`, `query_raw`, `device_type`, `geo_continent_name`, `geo_city_name`, `geo_country_iso_code`, `geo_region_name`, `geo_latitude`, `geo_longitude`, `click_link_url`, `click_link_index`, `click_clicked_object_id`, `click_title`, `click_query_id`, `click_context`, `query_type`, `query_total_banners`, `query_total_documents`, `query_response_time_ms`, `query_actual_query`, `query_quick_link_id`, `query_results_per_page`, `query_environment`, `query_request_type`, `query_banners`, `query_billable`, `application_name`, `application_version`, `http_status_code`, `aws_region`) FORMAT RowBinary ), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87a28ba in /usr/bin/clickhouse | |
| 1. DB::ReplicatedMergeTreeBlockOutputStream::commitPart(std::__1::shared_ptr<zkutil::ZooKeeper>&, std::__1::shared_ptr<DB::IMergeTreeDataPart>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0xfc17295 in /usr/bin/clickhouse | |
| 2. DB::ReplicatedMergeTreeBlockOutputStream::write(DB::Block const&) @ 0xfc133fd in /usr/bin/clickhouse | |
| 3. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 4. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 5. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 6. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0xfa175a2 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0xfa14109 in /usr/bin/clickhouse | |
| 9. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0xfa13334 in /usr/bin/clickhouse | |
| 10. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 11. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 13. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 14. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xeceefdd in /usr/bin/clickhouse | |
| 15. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf0a3879 in /usr/bin/clickhouse | |
| 16. DB::IBlockInputStream::read() @ 0xeccbbf5 in /usr/bin/clickhouse | |
| 17. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xecef236 in /usr/bin/clickhouse | |
| 18. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xf5ac165 in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0xfd2716f in /usr/bin/clickhouse | |
| 20. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0xfd2a889 in /usr/bin/clickhouse | |
| 21. DB::HTTPServerConnection::run() @ 0xfdae6f0 in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerConnection::start() @ 0x124232af in /usr/bin/clickhouse | |
| 23. Poco::Net::TCPServerDispatcher::run() @ 0x12424d3a in /usr/bin/clickhouse | |
| 24. Poco::PooledThread::run() @ 0x1255eaf9 in /usr/bin/clickhouse | |
| 25. Poco::ThreadImpl::runnableEntry(void*) @ 0x1255aaea in /usr/bin/clickhouse | |
| 26. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 27. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| 2021.06.24 11:23:07.963874 [ 9157 ] {462c4cfb-0fff-49d4-aed7-bafb08c68e6b} <Error> DynamicQueryHandler: Code: 244, e.displayText() = DB::Exception: Unexpected logical error while adding block 347087 with ID '202106_8149276849507687568_17514355474010194501': No node, path /clickhouse/analytics/tables/01/events_local/replicas/02/parts/202106_347087_347087_0, Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87a28ba in /usr/bin/clickhouse | |
| 1. DB::ReplicatedMergeTreeBlockOutputStream::commitPart(std::__1::shared_ptr<zkutil::ZooKeeper>&, std::__1::shared_ptr<DB::IMergeTreeDataPart>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0xfc17295 in /usr/bin/clickhouse | |
| 2. DB::ReplicatedMergeTreeBlockOutputStream::write(DB::Block const&) @ 0xfc133fd in /usr/bin/clickhouse | |
| 3. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 4. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 5. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 6. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0xfa175a2 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0xfa14109 in /usr/bin/clickhouse | |
| 9. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0xfa13334 in /usr/bin/clickhouse | |
| 10. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 11. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 13. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 14. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xeceefdd in /usr/bin/clickhouse | |
| 15. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf0a3879 in /usr/bin/clickhouse | |
| 16. DB::IBlockInputStream::read() @ 0xeccbbf5 in /usr/bin/clickhouse | |
| 17. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xecef236 in /usr/bin/clickhouse | |
| 18. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xf5ac165 in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0xfd2716f in /usr/bin/clickhouse | |
| 20. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0xfd2a889 in /usr/bin/clickhouse | |
| 21. DB::HTTPServerConnection::run() @ 0xfdae6f0 in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerConnection::start() @ 0x124232af in /usr/bin/clickhouse | |
| 23. Poco::Net::TCPServerDispatcher::run() @ 0x12424d3a in /usr/bin/clickhouse | |
| 24. Poco::PooledThread::run() @ 0x1255eaf9 in /usr/bin/clickhouse | |
| 25. Poco::ThreadImpl::runnableEntry(void*) @ 0x1255aaea in /usr/bin/clickhouse | |
| 26. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 27. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.5.6.6 (official build)) | |
| 2021.06.24 11:23:08.067357 [ 642 ] {} <Error> analytics.events_local_temp (89b20029-538f-434d-a1e3-e048226e872b): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 999, e.displayText() = Coordination::Exception: Can't get data for node /clickhouse/analytics/tables/01/events_local/replicas/02/log_pointer: node doesn't exist (No node), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87a28ba in /usr/bin/clickhouse | |
| 1. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error, int) @ 0x100ff595 in /usr/bin/clickhouse | |
| 2. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error) @ 0x100ff822 in /usr/bin/clickhouse | |
| 3. zkutil::ZooKeeper::get(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Stat*, std::__1::shared_ptr<Poco::Event> const&) @ 0x10106d12 in /usr/bin/clickhouse | |
| 4. DB::ReplicatedMergeTreeQueue::pullLogsToQueue(std::__1::shared_ptr<zkutil::ZooKeeper>, std::__1::function<void (Coordination::WatchResponse const&)>) @ 0xfc44167 in /usr/bin/clickhouse | |
| 5. DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0xf8945ca in /usr/bin/clickhouse | |
| 6. DB::BackgroundSchedulePoolTaskInfo::execute() @ 0xef0cc20 in /usr/bin/clickhouse | |
| 7. DB::BackgroundSchedulePool::threadFunction() @ 0xef0ec77 in /usr/bin/clickhouse | |
| 8. ? @ 0xef0f9f2 in /usr/bin/clickhouse | |
| 9. ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0x87e2b5f in /usr/bin/clickhouse | |
| 10. ? @ 0x87e6083 in /usr/bin/clickhouse | |
| 11. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 12. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.5.6.6 (official build)) | |
| 2021.06.24 11:23:08.840895 [ 654 ] {} <Error> analytics.events_local_temp (89b20029-538f-434d-a1e3-e048226e872b): void DB::StorageReplicatedMergeTree::mergeSelectingTask(): Code: 999, e.displayText() = Coordination::Exception: Can't get data for node /clickhouse/analytics/tables/01/events_local/replicas/02/log_pointer: node doesn't exist (No node), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87a28ba in /usr/bin/clickhouse | |
| 1. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error, int) @ 0x100ff595 in /usr/bin/clickhouse | |
| 2. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error) @ 0x100ff822 in /usr/bin/clickhouse | |
| 3. zkutil::ZooKeeper::get(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Stat*, std::__1::shared_ptr<Poco::Event> const&) @ 0x10106d12 in /usr/bin/clickhouse | |
| 4. DB::ReplicatedMergeTreeQueue::pullLogsToQueue(std::__1::shared_ptr<zkutil::ZooKeeper>, std::__1::function<void (Coordination::WatchResponse const&)>) @ 0xfc44167 in /usr/bin/clickhouse | |
| 5. DB::ReplicatedMergeTreeMergePredicate::ReplicatedMergeTreeMergePredicate(DB::ReplicatedMergeTreeQueue&, std::__1::shared_ptr<zkutil::ZooKeeper>&) @ 0xfc60e15 in /usr/bin/clickhouse | |
| 6. DB::StorageReplicatedMergeTree::mergeSelectingTask() @ 0xf895b4a in /usr/bin/clickhouse | |
| 7. DB::BackgroundSchedulePoolTaskInfo::execute() @ 0xef0cc20 in /usr/bin/clickhouse | |
| 8. DB::BackgroundSchedulePool::threadFunction() @ 0xef0ec77 in /usr/bin/clickhouse | |
| 9. ? @ 0xef0f9f2 in /usr/bin/clickhouse | |
| 10. ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0x87e2b5f in /usr/bin/clickhouse | |
| 11. ? @ 0x87e6083 in /usr/bin/clickhouse | |
| 12. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 13. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.5.6.6 (official build)) | |
| 2021.06.24 11:23:09.040671 [ 9157 ] {7dfbc288-d7e6-4ba9-b063-ed24558481df} <Error> executeQuery: Code: 244, e.displayText() = DB::Exception: Unexpected logical error while adding block 347089 with ID '202106_8149276849507687568_17514355474010194501': No node, path /clickhouse/analytics/tables/01/events_local/replicas/02/parts/202106_347089_347089_0 (version 21.5.6.6 (official build)) (from 172.31.18.125:56116) (in query: INSERT INTO events (`id`, `event_type`, `date`, `customer_id`, `engine_id`, `query`, `display_width_px`, `display_height_px`, `user_agent`, `referrer_url`, `referrer_title`, `browser_language`, `ip_address`, `page_number`, `host_name`, `session_id`, `query_session_id`, `query_raw`, `device_type`, `geo_continent_name`, `geo_city_name`, `geo_country_iso_code`, `geo_region_name`, `geo_latitude`, `geo_longitude`, `click_link_url`, `click_link_index`, `click_clicked_object_id`, `click_title`, `click_query_id`, `click_context`, `query_type`, `query_total_banners`, `query_total_documents`, `query_response_time_ms`, `query_actual_query`, `query_quick_link_id`, `query_results_per_page`, `query_environment`, `query_request_type`, `query_banners`, `query_billable`, `application_name`, `application_version`, `http_status_code`, `aws_region`) FORMAT RowBinary ), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87a28ba in /usr/bin/clickhouse | |
| 1. DB::ReplicatedMergeTreeBlockOutputStream::commitPart(std::__1::shared_ptr<zkutil::ZooKeeper>&, std::__1::shared_ptr<DB::IMergeTreeDataPart>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0xfc17295 in /usr/bin/clickhouse | |
| 2. DB::ReplicatedMergeTreeBlockOutputStream::write(DB::Block const&) @ 0xfc133fd in /usr/bin/clickhouse | |
| 3. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 4. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 5. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 6. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0xfa175a2 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0xfa14109 in /usr/bin/clickhouse | |
| 9. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0xfa13334 in /usr/bin/clickhouse | |
| 10. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 11. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 13. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 14. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xeceefdd in /usr/bin/clickhouse | |
| 15. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf0a3879 in /usr/bin/clickhouse | |
| 16. DB::IBlockInputStream::read() @ 0xeccbbf5 in /usr/bin/clickhouse | |
| 17. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xecef236 in /usr/bin/clickhouse | |
| 18. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xf5ac165 in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0xfd2716f in /usr/bin/clickhouse | |
| 20. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0xfd2a889 in /usr/bin/clickhouse | |
| 21. DB::HTTPServerConnection::run() @ 0xfdae6f0 in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerConnection::start() @ 0x124232af in /usr/bin/clickhouse | |
| 23. Poco::Net::TCPServerDispatcher::run() @ 0x12424d3a in /usr/bin/clickhouse | |
| 24. Poco::PooledThread::run() @ 0x1255eaf9 in /usr/bin/clickhouse | |
| 25. Poco::ThreadImpl::runnableEntry(void*) @ 0x1255aaea in /usr/bin/clickhouse | |
| 26. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 27. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| 2021.06.24 11:23:09.040927 [ 9157 ] {7dfbc288-d7e6-4ba9-b063-ed24558481df} <Error> DynamicQueryHandler: Code: 244, e.displayText() = DB::Exception: Unexpected logical error while adding block 347089 with ID '202106_8149276849507687568_17514355474010194501': No node, path /clickhouse/analytics/tables/01/events_local/replicas/02/parts/202106_347089_347089_0, Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87a28ba in /usr/bin/clickhouse | |
| 1. DB::ReplicatedMergeTreeBlockOutputStream::commitPart(std::__1::shared_ptr<zkutil::ZooKeeper>&, std::__1::shared_ptr<DB::IMergeTreeDataPart>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0xfc17295 in /usr/bin/clickhouse | |
| 2. DB::ReplicatedMergeTreeBlockOutputStream::write(DB::Block const&) @ 0xfc133fd in /usr/bin/clickhouse | |
| 3. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 4. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 5. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 6. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0xfa175a2 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0xfa14109 in /usr/bin/clickhouse | |
| 9. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0xfa13334 in /usr/bin/clickhouse | |
| 10. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 11. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 13. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 14. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xeceefdd in /usr/bin/clickhouse | |
| 15. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf0a3879 in /usr/bin/clickhouse | |
| 16. DB::IBlockInputStream::read() @ 0xeccbbf5 in /usr/bin/clickhouse | |
| 17. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xecef236 in /usr/bin/clickhouse | |
| 18. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xf5ac165 in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0xfd2716f in /usr/bin/clickhouse | |
| 20. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0xfd2a889 in /usr/bin/clickhouse | |
| 21. DB::HTTPServerConnection::run() @ 0xfdae6f0 in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerConnection::start() @ 0x124232af in /usr/bin/clickhouse | |
| 23. Poco::Net::TCPServerDispatcher::run() @ 0x12424d3a in /usr/bin/clickhouse | |
| 24. Poco::PooledThread::run() @ 0x1255eaf9 in /usr/bin/clickhouse | |
| 25. Poco::ThreadImpl::runnableEntry(void*) @ 0x1255aaea in /usr/bin/clickhouse | |
| 26. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 27. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.5.6.6 (official build)) | |
| 2021.06.24 11:23:09.068178 [ 639 ] {} <Error> analytics.events_local_temp (89b20029-538f-434d-a1e3-e048226e872b): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 999, e.displayText() = Coordination::Exception: Can't get data for node /clickhouse/analytics/tables/01/events_local/replicas/02/log_pointer: node doesn't exist (No node), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87a28ba in /usr/bin/clickhouse | |
| 1. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error, int) @ 0x100ff595 in /usr/bin/clickhouse | |
| 2. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error) @ 0x100ff822 in /usr/bin/clickhouse | |
| 3. zkutil::ZooKeeper::get(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Stat*, std::__1::shared_ptr<Poco::Event> const&) @ 0x10106d12 in /usr/bin/clickhouse | |
| 4. DB::ReplicatedMergeTreeQueue::pullLogsToQueue(std::__1::shared_ptr<zkutil::ZooKeeper>, std::__1::function<void (Coordination::WatchResponse const&)>) @ 0xfc44167 in /usr/bin/clickhouse | |
| 5. DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0xf8945ca in /usr/bin/clickhouse | |
| 6. DB::BackgroundSchedulePoolTaskInfo::execute() @ 0xef0cc20 in /usr/bin/clickhouse | |
| 7. DB::BackgroundSchedulePool::threadFunction() @ 0xef0ec77 in /usr/bin/clickhouse | |
| 8. ? @ 0xef0f9f2 in /usr/bin/clickhouse | |
| 9. ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0x87e2b5f in /usr/bin/clickhouse | |
| 10. ? @ 0x87e6083 in /usr/bin/clickhouse | |
| 11. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 12. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.5.6.6 (official build)) | |
| 2021.06.24 11:23:10.048611 [ 9157 ] {2c1746f7-8e68-44d4-8d20-697ffa7ddad0} <Warning> ClusterProxy::SelectStreamFactory: Local replica of shard 1 is stale (delay: 397041s.) | |
| 2021.06.24 11:23:10.069031 [ 629 ] {} <Error> analytics.events_local_temp (89b20029-538f-434d-a1e3-e048226e872b): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 999, e.displayText() = Coordination::Exception: Can't get data for node /clickhouse/analytics/tables/01/events_local/replicas/02/log_pointer: node doesn't exist (No node), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87a28ba in /usr/bin/clickhouse | |
| 1. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error, int) @ 0x100ff595 in /usr/bin/clickhouse | |
| 2. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error) @ 0x100ff822 in /usr/bin/clickhouse | |
| 3. zkutil::ZooKeeper::get(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Stat*, std::__1::shared_ptr<Poco::Event> const&) @ 0x10106d12 in /usr/bin/clickhouse | |
| 4. DB::ReplicatedMergeTreeQueue::pullLogsToQueue(std::__1::shared_ptr<zkutil::ZooKeeper>, std::__1::function<void (Coordination::WatchResponse const&)>) @ 0xfc44167 in /usr/bin/clickhouse | |
| 5. DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0xf8945ca in /usr/bin/clickhouse | |
| 6. DB::BackgroundSchedulePoolTaskInfo::execute() @ 0xef0cc20 in /usr/bin/clickhouse | |
| 7. DB::BackgroundSchedulePool::threadFunction() @ 0xef0ec77 in /usr/bin/clickhouse | |
| 8. ? @ 0xef0f9f2 in /usr/bin/clickhouse | |
| 9. ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0x87e2b5f in /usr/bin/clickhouse | |
| 10. ? @ 0x87e6083 in /usr/bin/clickhouse | |
| 11. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 12. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.5.6.6 (official build)) | |
| 2021.06.24 11:23:10.183808 [ 9157 ] {4ccc6b3b-3b80-46aa-94e3-8c4fa4d598ad} <Error> executeQuery: Code: 244, e.displayText() = DB::Exception: Unexpected logical error while adding block 347091 with ID '202106_12652627302720654630_16504372214202426708': No node, path /clickhouse/analytics/tables/01/events_local/replicas/02/parts/202106_347091_347091_0 (version 21.5.6.6 (official build)) (from 172.31.18.125:56128) (in query: INSERT INTO events (`id`, `event_type`, `date`, `customer_id`, `engine_id`, `query`, `display_width_px`, `display_height_px`, `user_agent`, `referrer_url`, `referrer_title`, `browser_language`, `ip_address`, `page_number`, `host_name`, `session_id`, `query_session_id`, `query_raw`, `device_type`, `geo_continent_name`, `geo_city_name`, `geo_country_iso_code`, `geo_region_name`, `geo_latitude`, `geo_longitude`, `click_link_url`, `click_link_index`, `click_clicked_object_id`, `click_title`, `click_query_id`, `click_context`, `query_type`, `query_total_banners`, `query_total_documents`, `query_response_time_ms`, `query_actual_query`, `query_quick_link_id`, `query_results_per_page`, `query_environment`, `query_request_type`, `query_banners`, `query_billable`, `application_name`, `application_version`, `http_status_code`, `aws_region`) FORMAT RowBinary ), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87a28ba in /usr/bin/clickhouse | |
| 1. DB::ReplicatedMergeTreeBlockOutputStream::commitPart(std::__1::shared_ptr<zkutil::ZooKeeper>&, std::__1::shared_ptr<DB::IMergeTreeDataPart>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0xfc17295 in /usr/bin/clickhouse | |
| 2. DB::ReplicatedMergeTreeBlockOutputStream::write(DB::Block const&) @ 0xfc133fd in /usr/bin/clickhouse | |
| 3. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 4. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 5. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 6. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0xfa175a2 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0xfa14109 in /usr/bin/clickhouse | |
| 9. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0xfa13334 in /usr/bin/clickhouse | |
| 10. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 11. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 13. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 14. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xeceefdd in /usr/bin/clickhouse | |
| 15. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf0a3879 in /usr/bin/clickhouse | |
| 16. DB::IBlockInputStream::read() @ 0xeccbbf5 in /usr/bin/clickhouse | |
| 17. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xecef236 in /usr/bin/clickhouse | |
| 18. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xf5ac165 in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0xfd2716f in /usr/bin/clickhouse | |
| 20. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0xfd2a889 in /usr/bin/clickhouse | |
| 21. DB::HTTPServerConnection::run() @ 0xfdae6f0 in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerConnection::start() @ 0x124232af in /usr/bin/clickhouse | |
| 23. Poco::Net::TCPServerDispatcher::run() @ 0x12424d3a in /usr/bin/clickhouse | |
| 24. Poco::PooledThread::run() @ 0x1255eaf9 in /usr/bin/clickhouse | |
| 25. Poco::ThreadImpl::runnableEntry(void*) @ 0x1255aaea in /usr/bin/clickhouse | |
| 26. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 27. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| 2021.06.24 11:23:10.184070 [ 9157 ] {4ccc6b3b-3b80-46aa-94e3-8c4fa4d598ad} <Error> DynamicQueryHandler: Code: 244, e.displayText() = DB::Exception: Unexpected logical error while adding block 347091 with ID '202106_12652627302720654630_16504372214202426708': No node, path /clickhouse/analytics/tables/01/events_local/replicas/02/parts/202106_347091_347091_0, Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87a28ba in /usr/bin/clickhouse | |
| 1. DB::ReplicatedMergeTreeBlockOutputStream::commitPart(std::__1::shared_ptr<zkutil::ZooKeeper>&, std::__1::shared_ptr<DB::IMergeTreeDataPart>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0xfc17295 in /usr/bin/clickhouse | |
| 2. DB::ReplicatedMergeTreeBlockOutputStream::write(DB::Block const&) @ 0xfc133fd in /usr/bin/clickhouse | |
| 3. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 4. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 5. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 6. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0xfa175a2 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0xfa14109 in /usr/bin/clickhouse | |
| 9. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0xfa13334 in /usr/bin/clickhouse | |
| 10. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 11. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 13. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 14. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xeceefdd in /usr/bin/clickhouse | |
| 15. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf0a3879 in /usr/bin/clickhouse | |
| 16. DB::IBlockInputStream::read() @ 0xeccbbf5 in /usr/bin/clickhouse | |
| 17. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xecef236 in /usr/bin/clickhouse | |
| 18. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xf5ac165 in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0xfd2716f in /usr/bin/clickhouse | |
| 20. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0xfd2a889 in /usr/bin/clickhouse | |
| 21. DB::HTTPServerConnection::run() @ 0xfdae6f0 in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerConnection::start() @ 0x124232af in /usr/bin/clickhouse | |
| 23. Poco::Net::TCPServerDispatcher::run() @ 0x12424d3a in /usr/bin/clickhouse | |
| 24. Poco::PooledThread::run() @ 0x1255eaf9 in /usr/bin/clickhouse | |
| 25. Poco::ThreadImpl::runnableEntry(void*) @ 0x1255aaea in /usr/bin/clickhouse | |
| 26. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 27. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.5.6.6 (official build)) | |
| 2021.06.24 11:23:10.298083 [ 9157 ] {c193776e-9048-40ac-89f2-d4ca8806a555} <Warning> ClusterProxy::SelectStreamFactory: Local replica of shard 1 is stale (delay: 397041s.) | |
| 2021.06.24 11:23:11.070224 [ 638 ] {} <Error> analytics.events_local_temp (89b20029-538f-434d-a1e3-e048226e872b): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 999, e.displayText() = Coordination::Exception: Can't get data for node /clickhouse/analytics/tables/01/events_local/replicas/02/log_pointer: node doesn't exist (No node), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87a28ba in /usr/bin/clickhouse | |
| 1. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error, int) @ 0x100ff595 in /usr/bin/clickhouse | |
| 2. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error) @ 0x100ff822 in /usr/bin/clickhouse | |
| 3. zkutil::ZooKeeper::get(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Stat*, std::__1::shared_ptr<Poco::Event> const&) @ 0x10106d12 in /usr/bin/clickhouse | |
| 4. DB::ReplicatedMergeTreeQueue::pullLogsToQueue(std::__1::shared_ptr<zkutil::ZooKeeper>, std::__1::function<void (Coordination::WatchResponse const&)>) @ 0xfc44167 in /usr/bin/clickhouse | |
| 5. DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0xf8945ca in /usr/bin/clickhouse | |
| 6. DB::BackgroundSchedulePoolTaskInfo::execute() @ 0xef0cc20 in /usr/bin/clickhouse | |
| 7. DB::BackgroundSchedulePool::threadFunction() @ 0xef0ec77 in /usr/bin/clickhouse | |
| 8. ? @ 0xef0f9f2 in /usr/bin/clickhouse | |
| 9. ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0x87e2b5f in /usr/bin/clickhouse | |
| 10. ? @ 0x87e6083 in /usr/bin/clickhouse | |
| 11. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 12. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.5.6.6 (official build)) | |
| 2021.06.24 11:23:11.292326 [ 9157 ] {268657e7-9ce1-4a64-a953-e162b6f31f21} <Error> executeQuery: Code: 244, e.displayText() = DB::Exception: Unexpected logical error while adding block 347093 with ID '202106_12652627302720654630_16504372214202426708': No node, path /clickhouse/analytics/tables/01/events_local/replicas/02/parts/202106_347093_347093_0 (version 21.5.6.6 (official build)) (from 172.31.18.125:56140) (in query: INSERT INTO events (`id`, `event_type`, `date`, `customer_id`, `engine_id`, `query`, `display_width_px`, `display_height_px`, `user_agent`, `referrer_url`, `referrer_title`, `browser_language`, `ip_address`, `page_number`, `host_name`, `session_id`, `query_session_id`, `query_raw`, `device_type`, `geo_continent_name`, `geo_city_name`, `geo_country_iso_code`, `geo_region_name`, `geo_latitude`, `geo_longitude`, `click_link_url`, `click_link_index`, `click_clicked_object_id`, `click_title`, `click_query_id`, `click_context`, `query_type`, `query_total_banners`, `query_total_documents`, `query_response_time_ms`, `query_actual_query`, `query_quick_link_id`, `query_results_per_page`, `query_environment`, `query_request_type`, `query_banners`, `query_billable`, `application_name`, `application_version`, `http_status_code`, `aws_region`) FORMAT RowBinary ), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87a28ba in /usr/bin/clickhouse | |
| 1. DB::ReplicatedMergeTreeBlockOutputStream::commitPart(std::__1::shared_ptr<zkutil::ZooKeeper>&, std::__1::shared_ptr<DB::IMergeTreeDataPart>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0xfc17295 in /usr/bin/clickhouse | |
| 2. DB::ReplicatedMergeTreeBlockOutputStream::write(DB::Block const&) @ 0xfc133fd in /usr/bin/clickhouse | |
| 3. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 4. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 5. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 6. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0xfa175a2 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0xfa14109 in /usr/bin/clickhouse | |
| 9. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0xfa13334 in /usr/bin/clickhouse | |
| 10. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 11. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 13. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 14. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xeceefdd in /usr/bin/clickhouse | |
| 15. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf0a3879 in /usr/bin/clickhouse | |
| 16. DB::IBlockInputStream::read() @ 0xeccbbf5 in /usr/bin/clickhouse | |
| 17. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xecef236 in /usr/bin/clickhouse | |
| 18. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xf5ac165 in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0xfd2716f in /usr/bin/clickhouse | |
| 20. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0xfd2a889 in /usr/bin/clickhouse | |
| 21. DB::HTTPServerConnection::run() @ 0xfdae6f0 in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerConnection::start() @ 0x124232af in /usr/bin/clickhouse | |
| 23. Poco::Net::TCPServerDispatcher::run() @ 0x12424d3a in /usr/bin/clickhouse | |
| 24. Poco::PooledThread::run() @ 0x1255eaf9 in /usr/bin/clickhouse | |
| 25. Poco::ThreadImpl::runnableEntry(void*) @ 0x1255aaea in /usr/bin/clickhouse | |
| 26. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 27. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| 2021.06.24 11:23:11.292591 [ 9157 ] {268657e7-9ce1-4a64-a953-e162b6f31f21} <Error> DynamicQueryHandler: Code: 244, e.displayText() = DB::Exception: Unexpected logical error while adding block 347093 with ID '202106_12652627302720654630_16504372214202426708': No node, path /clickhouse/analytics/tables/01/events_local/replicas/02/parts/202106_347093_347093_0, Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87a28ba in /usr/bin/clickhouse | |
| 1. DB::ReplicatedMergeTreeBlockOutputStream::commitPart(std::__1::shared_ptr<zkutil::ZooKeeper>&, std::__1::shared_ptr<DB::IMergeTreeDataPart>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0xfc17295 in /usr/bin/clickhouse | |
| 2. DB::ReplicatedMergeTreeBlockOutputStream::write(DB::Block const&) @ 0xfc133fd in /usr/bin/clickhouse | |
| 3. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 4. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 5. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 6. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0xfa175a2 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0xfa14109 in /usr/bin/clickhouse | |
| 9. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0xfa13334 in /usr/bin/clickhouse | |
| 10. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 11. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 13. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 14. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xeceefdd in /usr/bin/clickhouse | |
| 15. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf0a3879 in /usr/bin/clickhouse | |
| 16. DB::IBlockInputStream::read() @ 0xeccbbf5 in /usr/bin/clickhouse | |
| 17. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xecef236 in /usr/bin/clickhouse | |
| 18. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xf5ac165 in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0xfd2716f in /usr/bin/clickhouse | |
| 20. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0xfd2a889 in /usr/bin/clickhouse | |
| 21. DB::HTTPServerConnection::run() @ 0xfdae6f0 in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerConnection::start() @ 0x124232af in /usr/bin/clickhouse | |
| 23. Poco::Net::TCPServerDispatcher::run() @ 0x12424d3a in /usr/bin/clickhouse | |
| 24. Poco::PooledThread::run() @ 0x1255eaf9 in /usr/bin/clickhouse | |
| 25. Poco::ThreadImpl::runnableEntry(void*) @ 0x1255aaea in /usr/bin/clickhouse | |
| 26. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 27. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.5.6.6 (official build)) | |
| 2021.06.24 11:23:12.071108 [ 637 ] {} <Error> analytics.events_local_temp (89b20029-538f-434d-a1e3-e048226e872b): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 999, e.displayText() = Coordination::Exception: Can't get data for node /clickhouse/analytics/tables/01/events_local/replicas/02/log_pointer: node doesn't exist (No node), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87a28ba in /usr/bin/clickhouse | |
| 1. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error, int) @ 0x100ff595 in /usr/bin/clickhouse | |
| 2. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error) @ 0x100ff822 in /usr/bin/clickhouse | |
| 3. zkutil::ZooKeeper::get(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Stat*, std::__1::shared_ptr<Poco::Event> const&) @ 0x10106d12 in /usr/bin/clickhouse | |
| 4. DB::ReplicatedMergeTreeQueue::pullLogsToQueue(std::__1::shared_ptr<zkutil::ZooKeeper>, std::__1::function<void (Coordination::WatchResponse const&)>) @ 0xfc44167 in /usr/bin/clickhouse | |
| 5. DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0xf8945ca in /usr/bin/clickhouse | |
| 6. DB::BackgroundSchedulePoolTaskInfo::execute() @ 0xef0cc20 in /usr/bin/clickhouse | |
| 7. DB::BackgroundSchedulePool::threadFunction() @ 0xef0ec77 in /usr/bin/clickhouse | |
| 8. ? @ 0xef0f9f2 in /usr/bin/clickhouse | |
| 9. ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0x87e2b5f in /usr/bin/clickhouse | |
| 10. ? @ 0x87e6083 in /usr/bin/clickhouse | |
| 11. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 12. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.5.6.6 (official build)) | |
| 2021.06.24 11:23:12.308373 [ 9157 ] {3f2ff1c7-d1dd-4431-aff6-d99155f5e906} <Warning> ClusterProxy::SelectStreamFactory: Local replica of shard 1 is stale (delay: 397043s.) | |
| 2021.06.24 11:23:12.437180 [ 9157 ] {9bbb4251-9423-4d00-bc97-eb6ddda3cdd8} <Error> executeQuery: Code: 244, e.displayText() = DB::Exception: Unexpected logical error while adding block 347095 with ID '202106_1048910171137207163_8194778068576351470': No node, path /clickhouse/analytics/tables/01/events_local/replicas/02/parts/202106_347095_347095_0 (version 21.5.6.6 (official build)) (from 172.31.18.125:56152) (in query: INSERT INTO events (`id`, `event_type`, `date`, `customer_id`, `engine_id`, `query`, `display_width_px`, `display_height_px`, `user_agent`, `referrer_url`, `referrer_title`, `browser_language`, `ip_address`, `page_number`, `host_name`, `session_id`, `query_session_id`, `query_raw`, `device_type`, `geo_continent_name`, `geo_city_name`, `geo_country_iso_code`, `geo_region_name`, `geo_latitude`, `geo_longitude`, `click_link_url`, `click_link_index`, `click_clicked_object_id`, `click_title`, `click_query_id`, `click_context`, `query_type`, `query_total_banners`, `query_total_documents`, `query_response_time_ms`, `query_actual_query`, `query_quick_link_id`, `query_results_per_page`, `query_environment`, `query_request_type`, `query_banners`, `query_billable`, `application_name`, `application_version`, `http_status_code`, `aws_region`) FORMAT RowBinary ), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87a28ba in /usr/bin/clickhouse | |
| 1. DB::ReplicatedMergeTreeBlockOutputStream::commitPart(std::__1::shared_ptr<zkutil::ZooKeeper>&, std::__1::shared_ptr<DB::IMergeTreeDataPart>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0xfc17295 in /usr/bin/clickhouse | |
| 2. DB::ReplicatedMergeTreeBlockOutputStream::write(DB::Block const&) @ 0xfc133fd in /usr/bin/clickhouse | |
| 3. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 4. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 5. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 6. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0xfa175a2 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0xfa14109 in /usr/bin/clickhouse | |
| 9. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0xfa13334 in /usr/bin/clickhouse | |
| 10. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 11. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 13. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 14. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xeceefdd in /usr/bin/clickhouse | |
| 15. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf0a3879 in /usr/bin/clickhouse | |
| 16. DB::IBlockInputStream::read() @ 0xeccbbf5 in /usr/bin/clickhouse | |
| 17. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xecef236 in /usr/bin/clickhouse | |
| 18. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xf5ac165 in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0xfd2716f in /usr/bin/clickhouse | |
| 20. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0xfd2a889 in /usr/bin/clickhouse | |
| 21. DB::HTTPServerConnection::run() @ 0xfdae6f0 in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerConnection::start() @ 0x124232af in /usr/bin/clickhouse | |
| 23. Poco::Net::TCPServerDispatcher::run() @ 0x12424d3a in /usr/bin/clickhouse | |
| 24. Poco::PooledThread::run() @ 0x1255eaf9 in /usr/bin/clickhouse | |
| 25. Poco::ThreadImpl::runnableEntry(void*) @ 0x1255aaea in /usr/bin/clickhouse | |
| 26. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 27. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| 2021.06.24 11:23:12.437427 [ 9157 ] {9bbb4251-9423-4d00-bc97-eb6ddda3cdd8} <Error> DynamicQueryHandler: Code: 244, e.displayText() = DB::Exception: Unexpected logical error while adding block 347095 with ID '202106_1048910171137207163_8194778068576351470': No node, path /clickhouse/analytics/tables/01/events_local/replicas/02/parts/202106_347095_347095_0, Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87a28ba in /usr/bin/clickhouse | |
| 1. DB::ReplicatedMergeTreeBlockOutputStream::commitPart(std::__1::shared_ptr<zkutil::ZooKeeper>&, std::__1::shared_ptr<DB::IMergeTreeDataPart>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0xfc17295 in /usr/bin/clickhouse | |
| 2. DB::ReplicatedMergeTreeBlockOutputStream::write(DB::Block const&) @ 0xfc133fd in /usr/bin/clickhouse | |
| 3. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 4. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 5. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 6. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0xfa175a2 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0xfa14109 in /usr/bin/clickhouse | |
| 9. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0xfa13334 in /usr/bin/clickhouse | |
| 10. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 11. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 13. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 14. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xeceefdd in /usr/bin/clickhouse | |
| 15. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf0a3879 in /usr/bin/clickhouse | |
| 16. DB::IBlockInputStream::read() @ 0xeccbbf5 in /usr/bin/clickhouse | |
| 17. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xecef236 in /usr/bin/clickhouse | |
| 18. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xf5ac165 in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0xfd2716f in /usr/bin/clickhouse | |
| 20. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0xfd2a889 in /usr/bin/clickhouse | |
| 21. DB::HTTPServerConnection::run() @ 0xfdae6f0 in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerConnection::start() @ 0x124232af in /usr/bin/clickhouse | |
| 23. Poco::Net::TCPServerDispatcher::run() @ 0x12424d3a in /usr/bin/clickhouse | |
| 24. Poco::PooledThread::run() @ 0x1255eaf9 in /usr/bin/clickhouse | |
| 25. Poco::ThreadImpl::runnableEntry(void*) @ 0x1255aaea in /usr/bin/clickhouse | |
| 26. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 27. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.5.6.6 (official build)) | |
| 2021.06.24 11:23:12.529169 [ 9157 ] {9220e1ff-84dd-4ffb-b853-142caa24e314} <Warning> ClusterProxy::SelectStreamFactory: Local replica of shard 1 is stale (delay: 397043s.) | |
| 2021.06.24 11:23:13.072175 [ 643 ] {} <Error> analytics.events_local_temp (89b20029-538f-434d-a1e3-e048226e872b): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 999, e.displayText() = Coordination::Exception: Can't get data for node /clickhouse/analytics/tables/01/events_local/replicas/02/log_pointer: node doesn't exist (No node), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87a28ba in /usr/bin/clickhouse | |
| 1. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error, int) @ 0x100ff595 in /usr/bin/clickhouse | |
| 2. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error) @ 0x100ff822 in /usr/bin/clickhouse | |
| 3. zkutil::ZooKeeper::get(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Stat*, std::__1::shared_ptr<Poco::Event> const&) @ 0x10106d12 in /usr/bin/clickhouse | |
| 4. DB::ReplicatedMergeTreeQueue::pullLogsToQueue(std::__1::shared_ptr<zkutil::ZooKeeper>, std::__1::function<void (Coordination::WatchResponse const&)>) @ 0xfc44167 in /usr/bin/clickhouse | |
| 5. DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0xf8945ca in /usr/bin/clickhouse | |
| 6. DB::BackgroundSchedulePoolTaskInfo::execute() @ 0xef0cc20 in /usr/bin/clickhouse | |
| 7. DB::BackgroundSchedulePool::threadFunction() @ 0xef0ec77 in /usr/bin/clickhouse | |
| 8. ? @ 0xef0f9f2 in /usr/bin/clickhouse | |
| 9. ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0x87e2b5f in /usr/bin/clickhouse | |
| 10. ? @ 0x87e6083 in /usr/bin/clickhouse | |
| 11. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 12. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.5.6.6 (official build)) | |
| 2021.06.24 11:23:13.585229 [ 9157 ] {952d9df3-7c43-4789-82da-c78500e40326} <Error> executeQuery: Code: 244, e.displayText() = DB::Exception: Unexpected logical error while adding block 347097 with ID '202106_1048910171137207163_8194778068576351470': No node, path /clickhouse/analytics/tables/01/events_local/replicas/02/parts/202106_347097_347097_0 (version 21.5.6.6 (official build)) (from 172.31.18.125:56164) (in query: INSERT INTO events (`id`, `event_type`, `date`, `customer_id`, `engine_id`, `query`, `display_width_px`, `display_height_px`, `user_agent`, `referrer_url`, `referrer_title`, `browser_language`, `ip_address`, `page_number`, `host_name`, `session_id`, `query_session_id`, `query_raw`, `device_type`, `geo_continent_name`, `geo_city_name`, `geo_country_iso_code`, `geo_region_name`, `geo_latitude`, `geo_longitude`, `click_link_url`, `click_link_index`, `click_clicked_object_id`, `click_title`, `click_query_id`, `click_context`, `query_type`, `query_total_banners`, `query_total_documents`, `query_response_time_ms`, `query_actual_query`, `query_quick_link_id`, `query_results_per_page`, `query_environment`, `query_request_type`, `query_banners`, `query_billable`, `application_name`, `application_version`, `http_status_code`, `aws_region`) FORMAT RowBinary ), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87a28ba in /usr/bin/clickhouse | |
| 1. DB::ReplicatedMergeTreeBlockOutputStream::commitPart(std::__1::shared_ptr<zkutil::ZooKeeper>&, std::__1::shared_ptr<DB::IMergeTreeDataPart>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0xfc17295 in /usr/bin/clickhouse | |
| 2. DB::ReplicatedMergeTreeBlockOutputStream::write(DB::Block const&) @ 0xfc133fd in /usr/bin/clickhouse | |
| 3. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 4. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 5. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 6. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0xfa175a2 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0xfa14109 in /usr/bin/clickhouse | |
| 9. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0xfa13334 in /usr/bin/clickhouse | |
| 10. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 11. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 13. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 14. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xeceefdd in /usr/bin/clickhouse | |
| 15. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf0a3879 in /usr/bin/clickhouse | |
| 16. DB::IBlockInputStream::read() @ 0xeccbbf5 in /usr/bin/clickhouse | |
| 17. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xecef236 in /usr/bin/clickhouse | |
| 18. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xf5ac165 in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0xfd2716f in /usr/bin/clickhouse | |
| 20. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0xfd2a889 in /usr/bin/clickhouse | |
| 21. DB::HTTPServerConnection::run() @ 0xfdae6f0 in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerConnection::start() @ 0x124232af in /usr/bin/clickhouse | |
| 23. Poco::Net::TCPServerDispatcher::run() @ 0x12424d3a in /usr/bin/clickhouse | |
| 24. Poco::PooledThread::run() @ 0x1255eaf9 in /usr/bin/clickhouse | |
| 25. Poco::ThreadImpl::runnableEntry(void*) @ 0x1255aaea in /usr/bin/clickhouse | |
| 26. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 27. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| 2021.06.24 11:23:13.585502 [ 9157 ] {952d9df3-7c43-4789-82da-c78500e40326} <Error> DynamicQueryHandler: Code: 244, e.displayText() = DB::Exception: Unexpected logical error while adding block 347097 with ID '202106_1048910171137207163_8194778068576351470': No node, path /clickhouse/analytics/tables/01/events_local/replicas/02/parts/202106_347097_347097_0, Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87a28ba in /usr/bin/clickhouse | |
| 1. DB::ReplicatedMergeTreeBlockOutputStream::commitPart(std::__1::shared_ptr<zkutil::ZooKeeper>&, std::__1::shared_ptr<DB::IMergeTreeDataPart>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0xfc17295 in /usr/bin/clickhouse | |
| 2. DB::ReplicatedMergeTreeBlockOutputStream::write(DB::Block const&) @ 0xfc133fd in /usr/bin/clickhouse | |
| 3. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 4. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 5. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 6. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 7. DB::DistributedBlockOutputStream::writeToLocal(DB::Block const&, unsigned long) @ 0xfa175a2 in /usr/bin/clickhouse | |
| 8. DB::DistributedBlockOutputStream::writeAsync(DB::Block const&) @ 0xfa14109 in /usr/bin/clickhouse | |
| 9. DB::DistributedBlockOutputStream::write(DB::Block const&) @ 0xfa13334 in /usr/bin/clickhouse | |
| 10. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xf0a6bd2 in /usr/bin/clickhouse | |
| 11. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xf0b102b in /usr/bin/clickhouse | |
| 12. DB::SquashingBlockOutputStream::finalize() @ 0xf0b16ac in /usr/bin/clickhouse | |
| 13. DB::SquashingBlockOutputStream::writeSuffix() @ 0xf0b1749 in /usr/bin/clickhouse | |
| 14. DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*) @ 0xeceefdd in /usr/bin/clickhouse | |
| 15. DB::NullAndDoCopyBlockInputStream::readImpl() @ 0xf0a3879 in /usr/bin/clickhouse | |
| 16. DB::IBlockInputStream::read() @ 0xeccbbf5 in /usr/bin/clickhouse | |
| 17. void DB::copyDataImpl<std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&>(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::function<bool ()> const&, std::__1::function<void (DB::Block const&)> const&) @ 0xecef236 in /usr/bin/clickhouse | |
| 18. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) @ 0xf5ac165 in /usr/bin/clickhouse | |
| 19. DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0xfd2716f in /usr/bin/clickhouse | |
| 20. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0xfd2a889 in /usr/bin/clickhouse | |
| 21. DB::HTTPServerConnection::run() @ 0xfdae6f0 in /usr/bin/clickhouse | |
| 22. Poco::Net::TCPServerConnection::start() @ 0x124232af in /usr/bin/clickhouse | |
| 23. Poco::Net::TCPServerDispatcher::run() @ 0x12424d3a in /usr/bin/clickhouse | |
| 24. Poco::PooledThread::run() @ 0x1255eaf9 in /usr/bin/clickhouse | |
| 25. Poco::ThreadImpl::runnableEntry(void*) @ 0x1255aaea in /usr/bin/clickhouse | |
| 26. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 27. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.5.6.6 (official build)) | |
| 2021.06.24 11:23:13.842277 [ 637 ] {} <Error> analytics.events_local_temp (89b20029-538f-434d-a1e3-e048226e872b): void DB::StorageReplicatedMergeTree::mergeSelectingTask(): Code: 999, e.displayText() = Coordination::Exception: Can't get data for node /clickhouse/analytics/tables/01/events_local/replicas/02/log_pointer: node doesn't exist (No node), Stack trace (when copying this message, always include the lines below): | |
| 0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x87a28ba in /usr/bin/clickhouse | |
| 1. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error, int) @ 0x100ff595 in /usr/bin/clickhouse | |
| 2. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error) @ 0x100ff822 in /usr/bin/clickhouse | |
| 3. zkutil::ZooKeeper::get(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Stat*, std::__1::shared_ptr<Poco::Event> const&) @ 0x10106d12 in /usr/bin/clickhouse | |
| 4. DB::ReplicatedMergeTreeQueue::pullLogsToQueue(std::__1::shared_ptr<zkutil::ZooKeeper>, std::__1::function<void (Coordination::WatchResponse const&)>) @ 0xfc44167 in /usr/bin/clickhouse | |
| 5. DB::ReplicatedMergeTreeMergePredicate::ReplicatedMergeTreeMergePredicate(DB::ReplicatedMergeTreeQueue&, std::__1::shared_ptr<zkutil::ZooKeeper>&) @ 0xfc60e15 in /usr/bin/clickhouse | |
| 6. DB::StorageReplicatedMergeTree::mergeSelectingTask() @ 0xf895b4a in /usr/bin/clickhouse | |
| 7. DB::BackgroundSchedulePoolTaskInfo::execute() @ 0xef0cc20 in /usr/bin/clickhouse | |
| 8. DB::BackgroundSchedulePool::threadFunction() @ 0xef0ec77 in /usr/bin/clickhouse | |
| 9. ? @ 0xef0f9f2 in /usr/bin/clickhouse | |
| 10. ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0x87e2b5f in /usr/bin/clickhouse | |
| 11. ? @ 0x87e6083 in /usr/bin/clickhouse | |
| 12. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so | |
| 13. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so | |
| (version 21.5.6.6 (official build)) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment