
{"id":8116,"date":"2025-08-19T10:46:41","date_gmt":"2025-08-19T09:46:41","guid":{"rendered":"https:\/\/dasini.net\/blog\/?p=8116"},"modified":"2025-08-19T10:46:43","modified_gmt":"2025-08-19T09:46:43","slug":"archive-smarter-query-faster-unlocking-mysql-performance-with-heatwave","status":"publish","type":"post","link":"https:\/\/dasini.net\/blog\/2025\/08\/19\/archive-smarter-query-faster-unlocking-mysql-performance-with-heatwave\/","title":{"rendered":"Archive Smarter, Query Faster: Unlocking MySQL Performance with HeatWave"},"content":{"rendered":"\n<p>Managing large volumes of data is a challenge every organization faces as applications grow. For MySQL users, this often means deciding what to do with historical or less frequently accessed data: keep it in production tables at the cost of performance, or archive it and lose the ability to query it efficiently. <br>Traditionally, archiving has been a trade-off \u2014 helpful for keeping databases lean, but limiting when developers or analysts need to run queries across years of historical records.<\/p>\n\n\n\n<p><strong><a href=\"https:\/\/www.oracle.com\/heatwave\/\" target=\"_blank\" rel=\"noopener\" title=\"MySQL HeatWave - Improve MySQL query performance by orders of magnitude and get real-time analytics on your transactional data\">HeatWave MySQL<\/a> changes this equation<\/strong>!<\/p>\n\n\n\n<p>By combining a <strong>high-performance in-memory query accelerator<\/strong> with a <strong>fully managed MySQL database service<\/strong>, HeatWave makes it possible to archive vast amounts of data while still enabling fast, interactive analytics. Instead of moving archived data into a separate system or relying on slow queries, developers and DBAs can continue to use familiar SQL while taking advantage of HeatWave\u2019s speed and scale.<\/p>\n\n\n\n<p>In this article, we\u2019ll explore how to use HeatWave to archive MySQL data effectively and then run accelerated queries on that archived data \u2014 <strong>without compromising performance<\/strong>. Whether you\u2019re managing billions of rows or just planning for future growth, this approach can help simplify your architecture while delivering the best of both worlds: efficient data storage and lightning-fast analytics.<\/p>\n\n\n<div class=\"wp-block-image is-style-default\">\n<figure data-wp-context=\"{&quot;imageId&quot;:&quot;69e752e0a245c&quot;}\" data-wp-interactive=\"core\/image\" data-wp-key=\"69e752e0a245c\" class=\"aligncenter size-thumbnail wp-lightbox-container\"><img loading=\"lazy\" decoding=\"async\" width=\"830\" height=\"271\" data-wp-class--hide=\"state.isContentHidden\" data-wp-class--show=\"state.isContentVisible\" data-wp-init=\"callbacks.setButtonStyles\" data-wp-on--click=\"actions.showLightbox\" data-wp-on--load=\"callbacks.setButtonStyles\" data-wp-on-window--resize=\"callbacks.setButtonStyles\" src=\"https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2024\/12\/HeatWave_logo.png?resize=150%2C150&amp;ssl=1\" alt=\"HeatWave\" class=\"wp-image-7222\" srcset=\"https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2024\/12\/HeatWave_logo.png?w=830&amp;ssl=1 830w, https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2024\/12\/HeatWave_logo.png?resize=300%2C98&amp;ssl=1 300w, https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2024\/12\/HeatWave_logo.png?resize=800%2C261&amp;ssl=1 800w, https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2024\/12\/HeatWave_logo.png?resize=768%2C251&amp;ssl=1 768w\" sizes=\"auto, (max-width: 830px) 100vw, 830px\" \/><button\n\t\t\tclass=\"lightbox-trigger\"\n\t\t\ttype=\"button\"\n\t\t\taria-haspopup=\"dialog\"\n\t\t\taria-label=\"Agrandir\"\n\t\t\tdata-wp-init=\"callbacks.initTriggerButton\"\n\t\t\tdata-wp-on--click=\"actions.showLightbox\"\n\t\t\tdata-wp-style--right=\"state.imageButtonRight\"\n\t\t\tdata-wp-style--top=\"state.imageButtonTop\"\n\t\t>\n\t\t\t<svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"12\" height=\"12\" fill=\"none\" viewBox=\"0 0 12 12\">\n\t\t\t\t<path fill=\"#fff\" d=\"M2 0a2 2 0 0 0-2 2v2h1.5V2a.5.5 0 0 1 .5-.5h2V0H2Zm2 10.5H2a.5.5 0 0 1-.5-.5V8H0v2a2 2 0 0 0 2 2h2v-1.5ZM8 12v-1.5h2a.5.5 0 0 0 .5-.5V8H12v2a2 2 0 0 1-2 2H8Zm2-12a2 2 0 0 1 2 2v2h-1.5V2a.5.5 0 0 0-.5-.5H8V0h2Z\" \/>\n\t\t\t<\/svg>\n\t\t<\/button><\/figure>\n<\/div>\n\n\n<h2 class=\"wp-block-heading\">Context<\/h2>\n\n\n\n<p>The database has been collecting and collecting daily time-stamped data for years.To optimize performance, we want to keep only the current year\u2019s data plus the full year before it. All older data should be archived, yet still available for querying with very low latency.<br>Let\u2019s see how HeatWave can help us archive efficiently while still running lightning-fast queries on historical data.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity is-style-dots\"\/>\n\n\n\n<p>I&rsquo;m using <a href=\"https:\/\/www.oracle.com\/heatwave\/\" target=\"_blank\" rel=\"noopener\" title=\"HeatWave\"><strong>HeatWave<\/strong><\/a> <strong>8.4.6<\/strong> on <a href=\"https:\/\/www.oracle.com\/cloud\/\" target=\"_blank\" rel=\"noopener\" title=\"Oracle Cloud Infrastructure\">Oracle Cloud Infrastructure<\/a> (OCI):<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"sql\" class=\"language-sql\">SQL&gt; \n-- HeatWave MySQL server version\nSHOW VARIABLES WHERE Variable_name IN ('version_comment', 'version');\n+-----------------+--------------------------+\n| Variable_name   | Value                    |\n+-----------------+--------------------------+\n| version         | 8.4.6-cloud              |\n| version_comment | MySQL Enterprise - Cloud |\n+-----------------+--------------------------+<\/code><\/pre>\n\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p>with a 1 node HeatWave Cluster enable.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full is-style-default\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" width=\"279\" height=\"112\" src=\"https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2025\/08\/HW_cluster_active.png?resize=279%2C112&#038;ssl=1\" alt=\"HeatWave Cluster Enable\" class=\"wp-image-8133\"\/><\/figure>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"sql\" class=\"language-sql\">SQL&gt; \n-- HeatWave cluster node number \/ HeatWave Cluster status \nSHOW STATUS WHERE Variable_name IN ('rapid_cluster_ready_number', 'rapid_cluster_status');\n+----------------------------+-------+\n| Variable_name              | Value |\n+----------------------------+-------+\n| rapid_cluster_ready_number | 1     |\n| rapid_cluster_status       | ON    |\n+----------------------------+-------+<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-image size-full\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" width=\"330\" height=\"209\" src=\"https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2025\/08\/HW_cluster_details1.png?resize=330%2C209&#038;ssl=1\" alt=\"HeatWave Cluster Information\" class=\"wp-image-8134\" srcset=\"https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2025\/08\/HW_cluster_details1.png?w=330&amp;ssl=1 330w, https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2025\/08\/HW_cluster_details1.png?resize=300%2C190&amp;ssl=1 300w\" sizes=\"auto, (max-width: 330px) 100vw, 330px\" \/><\/figure>\n\n\n\n<div style=\"height:25px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p>We\u2019ll illustrate this article\u2019s examples using a dataset from <a href=\"undefined\" target=\"_blank\" rel=\"noopener\" title=\"Time Series Datasets\">Kaggle<\/a>.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"bash\" class=\"language-bash\">$ head -n 5 daily-minimum-temperatures-in-me.csv\nDate,Daily minimum temperatures\n1\/1\/1981,20.7\n1\/2\/1981,17.9\n1\/3\/1981,18.8\n1\/4\/1981,14.6<\/code><\/pre>\n\n\n\n<div style=\"height:25px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p>After some transformation, the data are stored in a partitioned table called:  <em>min_temp<\/em><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"sql\" class=\"language-sql\">SQL &gt; \nSHOW CREATE TABLE min_temp\\G\n*************************** 1. row ***************************\n       Table: min_temp\nCreate Table: CREATE TABLE `min_temp` (\n  `d` date DEFAULT NULL,\n  `temp` decimal(3,1) DEFAULT NULL\n) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci\n\/*!50100 PARTITION BY RANGE (year(`d`))\n(PARTITION p1981 VALUES LESS THAN (1982) ENGINE = InnoDB,\n PARTITION p1982 VALUES LESS THAN (1983) ENGINE = InnoDB,\n PARTITION p1983 VALUES LESS THAN (1984) ENGINE = InnoDB,\n PARTITION p1984 VALUES LESS THAN (1985) ENGINE = InnoDB,\n PARTITION p1985 VALUES LESS THAN (1986) ENGINE = InnoDB,\n PARTITION p1986 VALUES LESS THAN (1987) ENGINE = InnoDB,\n PARTITION p1987 VALUES LESS THAN (1988) ENGINE = InnoDB,\n PARTITION p1988 VALUES LESS THAN (1989) ENGINE = InnoDB,\n PARTITION p1989 VALUES LESS THAN (1990) ENGINE = InnoDB,\n PARTITION p1990 VALUES LESS THAN (1991) ENGINE = InnoDB) *\/<\/code><\/pre>\n\n\n\n<div style=\"height:10px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p>It contains time series data with daily temperature measurements:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"SQL\" class=\"language-SQL\"> SQL > \nSELECT * FROM min_temp LIMIT 3;\n+------------+------+\n| d          | temp |\n+------------+------+\n| 1981-01-01 | 20.7 |\n| 1981-01-02 | 17.9 |\n| 1981-01-03 | 18.8 |\n+------------+------+\n\n\nSELECT YEAR(d) AS temp_year FROM min_temp GROUP BY temp_year;\n+-----------+\n| temp_year |\n+-----------+\n|      1981 |\n|      1982 |\n|      1983 |\n|      1984 |\n|      1985 |\n|      1986 |\n|      1987 |\n|      1988 |\n|      1989 |\n|      1990 |\n+-----------+\n10 rows in set (0.0014 sec)<\/code><\/pre>\n\n\n\n<div style=\"height:10px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p>Use the <code><em><a href=\"https:\/\/dev.mysql.com\/doc\/refman\/8.4\/en\/information-schema-partitions-table.html\" target=\"_blank\" rel=\"noopener\" title=\"The INFORMATION_SCHEMA PARTITIONS Table\">INFORMATION_SCHEMA.PARTITIONS<\/a><\/em><\/code> table to inspect partition information:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"SQL\" class=\"language-SQL\">SELECT PARTITION_NAME, PARTITION_ORDINAL_POSITION, PARTITION_DESCRIPTION, TABLE_ROWS \nFROM INFORMATION_SCHEMA.PARTITIONS \nWHERE TABLE_NAME = 'min_temp';\n+----------------+----------------------------+-----------------------+------------+\n| PARTITION_NAME | PARTITION_ORDINAL_POSITION | PARTITION_DESCRIPTION | TABLE_ROWS |\n+----------------+----------------------------+-----------------------+------------+\n| p1981          |                          1 | 1982                  |        365 |\n| p1982          |                          2 | 1983                  |        365 |\n| p1983          |                          3 | 1984                  |        365 |\n| p1984          |                          4 | 1985                  |        365 |\n| p1985          |                          5 | 1986                  |        365 |\n| p1986          |                          6 | 1987                  |        365 |\n| p1987          |                          7 | 1988                  |        365 |\n| p1988          |                          8 | 1989                  |        365 |\n| p1989          |                          9 | 1990                  |        365 |\n| p1990          |                         10 | 1991                  |        365 |\n+----------------+----------------------------+-----------------------+------------+<\/code><\/pre>\n\n\n\n<div style=\"height:50px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h2 class=\"wp-block-heading\">Exchanging Partitions and Subpartitions with Tables<\/h2>\n\n\n\n<p>In our case, a practical approach to archiving is to convert the partitions containing old data into regular tables. With {HeatWave} MySQL, this process is straightforward. <br>Let\u2019s take a look at how it works.<\/p>\n\n\n\n<p>In MySQL, it is possible to exchange a table partition or subpartition with a table using<a href=\"https:\/\/dev.mysql.com\/doc\/refman\/8.4\/en\/partitioning-management-exchange.html\" target=\"_blank\" rel=\"noopener\" title=\"Exchange a table partition or subpartition with a table using ALTER TABLE\">&nbsp;<code>ALTER TABLE<\/code><\/a>. There are some requirements, so you may want to read the <a href=\"https:\/\/dev.mysql.com\/doc\/refman\/8.4\/en\/partitioning-management-exchange.html\" target=\"_blank\" rel=\"noopener\" title=\"Exchanging Partitions and Subpartitions with Tables\">documentation<\/a>, but in short we first need to create a similar table than the one that contains the partitions. But this table must not be partitioned. Then we&rsquo;ll be able to transfer the data from the partition to this new table. <\/p>\n\n\n\n<p>In MySQL, you can use the <a href=\"https:\/\/dev.mysql.com\/doc\/refman\/8.4\/en\/partitioning-management-exchange.html\" target=\"_blank\" rel=\"noopener\" title=\"Exchange a table partition or subpartition with a table using ALTER TABLE\"><code>ALTER TABLE<\/code><\/a> command to exchange a partition or subpartition with a regular table. There are a few requirements to meet \u2014 so it\u2019s worth reviewing the <a href=\"https:\/\/dev.mysql.com\/doc\/refman\/8.4\/en\/partitioning-management-exchange.html\" target=\"_blank\" rel=\"noopener\" title=\"Exchanging Partitions and Subpartitions with Tables\">documentation<\/a> \u2014 but in short, the process starts by creating a table with the same structure as the partitioned one, except without partitions. Once that table is ready, you can transfer the data from the partition into it.<\/p>\n\n\n\n<p>To create the table (<em><code>min_temp_1981<\/code><\/em>) that will store the  data from partition <em>p1981<\/em>, you can do, a <a href=\"https:\/\/dev.mysql.com\/doc\/refman\/8.4\/en\/create-table-select.html\" target=\"_blank\" rel=\"noopener\" title=\"CREATE TABLE ... SELECT Statement\">CREATE&#8230; SELECT<\/a>:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"sql\" class=\"language-sql\">SQL > \nCREATE TABLE min_temp_1981 SELECT * FROM min_temp WHERE NULL;\nQuery OK, 0 rows affected (0.0126 sec)\n\nRecords: 0  Duplicates: 0  Warnings: 0\n\n\n-- No data was copied\nSELECT * FROM min_temp_1981 LIMIT 3;\nEmpty set (0.0005 sec)\n\n\nSHOW CREATE TABLE min_temp_1981\\G\n*************************** 1. row ***************************\n       Table: min_temp_1981\nCreate Table: CREATE TABLE `min_temp_1981` (\n  `d` date DEFAULT NULL,\n  `temp` decimal(3,1) DEFAULT NULL\n) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci<\/code><\/pre>\n\n\n\n<div style=\"height:25px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p>Then transfer the data, from partition <em>p1981<\/em> to table <em><code>min_temp_1981<\/code><\/em>, using <a href=\"https:\/\/dev.mysql.com\/doc\/refman\/8.4\/en\/partitioning-management-exchange.html\" target=\"_blank\" rel=\"noopener\" title=\"Exchanging Partitions and Subpartitions with Tables\">ALTER TABLE&#8230; EXCHANGE PARTITION&#8230; WITH TABLE<\/a>:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"sql\" class=\"language-sql\">SQL &gt; \nALTER TABLE min_temp EXCHANGE PARTITION p1981 WITH TABLE min_temp_1981;\nQuery OK, 0 rows affected (0.0181 sec)\n\n\nSELECT * FROM min_temp_1981 LIMIT 3;\n+------------+------+\n| d          | temp |\n+------------+------+\n| 1981-01-01 | 20.7 |\n| 1981-01-02 | 17.9 |\n| 1981-01-03 | 18.8 |\n+------------+------+\n\n\nSELECT YEAR(d) AS temp_year FROM min_temp_1981 GROUP BY temp_year;\n+-----------+\n| temp_year |\n+-----------+\n|      1981 |\n+-----------+<\/code><\/pre>\n\n\n\n<p>All the data from partition <em>p1981<\/em> is now in the table <em>min_temp_1981<\/em>.<\/p>\n\n\n\n<div style=\"height:25px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p>Like expected, in the partitioned table, the \u00ab\u00a01981 data\u00a0\u00bb is not there anymore:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"sql\" class=\"language-sql\">SQL &gt; \nSELECT * FROM min_temp LIMIT 3;\n+------------+------+\n| d          | temp |\n+------------+------+\n| 1982-01-01 | 17.0 |\n| 1982-01-02 | 15.0 |\n| 1982-01-03 | 13.5 |\n+------------+------+\n\n\nSELECT YEAR(d) AS temp_year FROM min_temp GROUP BY temp_year ;\n+-----------+\n| temp_year |\n+-----------+\n|      1982 |\n|      1983 |\n|      1984 |\n|      1985 |\n|      1986 |\n|      1987 |\n|      1988 |\n|      1989 |\n|      1990 |\n+-----------+\n9 rows in set (0.0014 sec)<\/code><\/pre>\n\n\n\n<div style=\"height:25px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p>But the partition is still there, and it is now empty (TABLE_ROWS=0):<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"sql\" class=\"language-sql\">SQL> \nSELECT PARTITION_NAME, PARTITION_ORDINAL_POSITION, PARTITION_DESCRIPTION, TABLE_ROWS \nFROM INFORMATION_SCHEMA.PARTITIONS \nWHERE TABLE_NAME = 'min_temp';\n+----------------+----------------------------+-----------------------+------------+\n| PARTITION_NAME | PARTITION_ORDINAL_POSITION | PARTITION_DESCRIPTION | TABLE_ROWS |\n+----------------+----------------------------+-----------------------+------------+\n| p1981          |                          1 | 1982                  |          0 |\n| p1982          |                          2 | 1983                  |        365 |\n| p1983          |                          3 | 1984                  |        365 |\n| p1984          |                          4 | 1985                  |        365 |\n| p1985          |                          5 | 1986                  |        365 |\n| p1986          |                          6 | 1987                  |        365 |\n| p1987          |                          7 | 1988                  |        365 |\n| p1988          |                          8 | 1989                  |        365 |\n| p1989          |                          9 | 1990                  |        365 |\n| p1990          |                         10 | 1991                  |        365 |\n+----------------+----------------------------+-----------------------+------------+<\/code><\/pre>\n\n\n\n<div style=\"height:100px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h2 class=\"wp-block-heading\">Load data into the HeatWave Cluster<\/h2>\n\n\n\n<p>In order to have lightning-fast queries, the archived data must be loaded into the <a href=\"https:\/\/docs.oracle.com\/en-us\/iaas\/mysql-database\/doc\/heatwave.html\" target=\"_blank\" rel=\"noopener\" title=\"HeatWave Cluster\">HeatWave cluster<\/a>.<\/p>\n\n\n\n<figure class=\"wp-block-pullquote has-text-align-center\" style=\"border-width:1px;border-radius:50px;font-size:16px\"><blockquote><p>HeatWave cluster provides a distributed, scalable, shared-nothing, in-memory, hybrid columnar, query processing engine. It can accelerate analytic queries, query external data stored in object storage, and perform machine learning.<\/p><\/blockquote><\/figure>\n\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p>To successfully load a table into a HeatWave cluster, you must first ensure it has a <strong>primary key<\/strong>. If the table you are trying to archive lacks a primary key, the service will return an <strong>\u00ab\u00a0<em>Unable to load table without primary key<\/em>\u00ab\u00a0<\/strong> error:<\/p>\n\n\n\n<pre title=\"Add a primary key\" class=\"wp-block-code\"><code lang=\"sql\" class=\"language-sql\">SQL > \n-- Add a primary key\nALTER TABLE min_temp_1981 \n    ADD COLUMN id_min_temp_1981 INT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY FIRST;\nQuery OK, 0 rows affected (0.0698 sec)\n\nRecords: 0  Duplicates: 0  Warnings: 0\n\n\nSHOW CREATE TABLE min_temp_1981\\G\n*************************** 1. row ***************************\n       Table: min_temp_1981\nCreate Table: CREATE TABLE `min_temp_1981` (\n  `id_min_temp_1981` int unsigned NOT NULL AUTO_INCREMENT,\n  `d` date DEFAULT NULL,\n  `temp` decimal(3,1) DEFAULT NULL,\n  PRIMARY KEY (`id_min_temp_1981`)\n) ENGINE=InnoDB AUTO_INCREMENT=366 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci\n\n\nSELECT * FROM min_temp_1981 LIMIT 3;\n+------------------+------------+------+\n| id_min_temp_1981 | d          | temp |\n+------------------+------------+------+\n|                1 | 1981-01-01 | 20.7 |\n|                2 | 1981-01-02 | 17.9 |\n|                3 | 1981-01-03 | 18.8 |\n+------------------+------------+------+<\/code><\/pre>\n\n\n\n<div style=\"height:25px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p>Now we can load the data into the HeatWave Cluster, using <a href=\"https:\/\/dev.mysql.com\/doc\/heatwave\/en\/mys-hw-auto-parallel-load-syntax.html\" target=\"_blank\" rel=\"noopener\" title=\"Load Data Using Auto Parallel Load\"><strong>sys.heatwave_load<\/strong><\/a>:<\/p>\n\n\n\n<pre title=\"Load Data Using Auto Parallel Load\" class=\"wp-block-code\"><code lang=\"sql\" class=\"language-sql\">SQL> \nCALL sys.heatwave_load(JSON_ARRAY(\"temp_archiving\"), JSON_OBJECT('include_list', JSON_ARRAY('temp_archiving.min_temp_1981')));\n\n\n+------------------------------------------+\n| INITIALIZING HEATWAVE AUTO PARALLEL LOAD |\n+------------------------------------------+\n| Version: 3.21                            |\n|                                          |\n| Load Mode: normal                        |\n| Load Policy: disable_unsupported_columns |\n| Output Mode: normal                      |\n|                                          |\n+------------------------------------------+\n6 rows in set (0.0175 sec)\n\n+------------------------------------------------------------------------+\n| OFFLOAD ANALYSIS                                                       |\n+------------------------------------------------------------------------+\n| Verifying input schemas: 1                                             |\n| User excluded items: 0                                                 |\n|                                                                        |\n| SCHEMA                       OFFLOADABLE    OFFLOADABLE     SUMMARY OF |\n| NAME                              TABLES        COLUMNS     ISSUES     |\n| ------                       -----------    -----------     ---------- |\n| `temp_archiving`                       1              3                |\n|                                                                        |\n| Total offloadable schemas: 1                                           |\n|                                                                        |\n+------------------------------------------------------------------------+\n10 rows in set (0.0175 sec)\n\n+-----------------------------------------------------------------------------------------------------------------------------+\n| CAPACITY ESTIMATION                                                                                                         |\n+-----------------------------------------------------------------------------------------------------------------------------+\n| Default encoding for string columns: VARLEN (unless specified in the schema)                                                |\n| Estimating memory footprint for 1 schema(s)                                                                                 |\n|                                                                                                                             |\n|                                TOTAL       ESTIMATED       ESTIMATED       TOTAL     DICTIONARY      VARLEN       ESTIMATED |\n| SCHEMA                   OFFLOADABLE   HEATWAVE NODE      MYSQL NODE      STRING        ENCODED     ENCODED            LOAD |\n| NAME                          TABLES       FOOTPRINT       FOOTPRINT     COLUMNS        COLUMNS     COLUMNS            TIME |\n| ------                   -----------       ---------       ---------     -------     ----------     -------       --------- |\n| `temp_archiving`                   1        3.27 MiB      256.00 KiB           0              0           0          1.00 s |\n|                                                                                                                             |\n| Sufficient MySQL host memory available to load all tables.                                                                  |\n| Sufficient HeatWave cluster memory available to load all tables.                                                            |\n|                                                                                                                             |\n+-----------------------------------------------------------------------------------------------------------------------------+\n12 rows in set (0.0175 sec)\n\n+---------------------------------------------------------------------------------------------------------------------------------------+\n| EXECUTING LOAD SCRIPT                                                                                                                 |\n+---------------------------------------------------------------------------------------------------------------------------------------+\n| HeatWave Load script generated                                                                                                        |\n|   Retrieve load script containing 3 generated DDL command(s) using the query below:                                                   |\n| Deprecation Notice: \"heatwave_load_report\" will be deprecated, please switch to \"heatwave_autopilot_report\"                           |\n|   SELECT log->>\"$.sql\" AS \"Load Script\" FROM sys.heatwave_autopilot_report WHERE type = \"sql\" ORDER BY id;                            |\n|                                                                                                                                       |\n| Adjusting load parallelism dynamically per internal\/external table.                                                                   |\n| Using current parallelism of 4 thread(s) as maximum for internal tables.                                                              |\n|                                                                                                                                       |\n| Warning: Executing the generated script may alter column definitions and secondary engine flags in the schema                         |\n|                                                                                                                                       |\n| Using SQL_MODE: ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTION |\n|                                                                                                                                       |\n| Proceeding to load 1 table(s) into HeatWave.                                                                                          |\n|                                                                                                                                       |\n| Applying changes will take approximately 113.00 ms                                                                                    |\n|                                                                                                                                       |\n+---------------------------------------------------------------------------------------------------------------------------------------+\n16 rows in set (0.0175 sec)\n\n+--------------------------------------------------+\n| TABLE LOAD                                       |\n+--------------------------------------------------+\n| TABLE (1 of 1): `temp_archiving`.`min_temp_1981` |\n| Commands executed successfully: 3 of 3           |\n| Warnings encountered: 0                          |\n| Table load succeeded!                            |\n|   Total columns loaded: 3                        |\n|   Table loaded using 1 thread(s)                 |\n|   Elapsed time: 22.70 s                          |\n|                                                  |\n+--------------------------------------------------+\n8 rows in set (0.0175 sec)\n\n+-------------------------------------------------------------------------------+\n| LOAD SUMMARY                                                                  |\n+-------------------------------------------------------------------------------+\n|                                                                               |\n| SCHEMA                          TABLES       TABLES      COLUMNS         LOAD |\n| NAME                            LOADED       FAILED       LOADED     DURATION |\n| ------                          ------       ------      -------     -------- |\n| `temp_archiving`                     1            0            3      22.70 s |\n|                                                                               |\n+-------------------------------------------------------------------------------+\n6 rows in set (0.0175 sec)\n\nQuery OK, 0 rows affected (0.0175 sec)<\/code><\/pre>\n\n\n\n<div style=\"height:25px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p>You can monitor the process using the <code><em><a href=\"https:\/\/dev.mysql.com\/doc\/heatwave\/en\/mys-hw-rpd-tables-table.html\" target=\"_blank\" rel=\"noopener\" title=\"The rpd_tables Table\">rpd_tables<\/a><\/em><\/code> &amp; <a href=\"https:\/\/dev.mysql.com\/doc\/heatwave\/en\/mys-hw-rpd-table-id-table.html\" target=\"_blank\" rel=\"noopener\" title=\"provides the ID, name, and schema of the tables loaded in MySQL HeatWave\">rpd_table_id<\/a> tables:<\/p>\n\n\n\n<pre title=\"Monitor the load progress\" class=\"wp-block-code\"><code lang=\"sql\" class=\"language-sql\">SQL> \nSELECT name, load_progress, load_status, query_count \nFROM performance_schema.rpd_tables JOIN performance_schema.rpd_table_id USING(id) \nWHERE name LIKE 'temp_archiving%' \nORDER BY name\\G\n*************************** 1. row ***************************\n         name: temp_archiving.min_temp_1981\nload_progress: 100\n  load_status: AVAIL_RPDGSTABSTATE\n  query_count: 0\n\n\nDESC performance_schema.rpd_table_id;\n+-------------+--------------+------+-----+---------+-------+\n| Field       | Type         | Null | Key | Default | Extra |\n+-------------+--------------+------+-----+---------+-------+\n| ID          | int unsigned | NO   |     | NULL    |       |\n| NAME        | varchar(129) | YES  |     | NULL    |       |\n| SCHEMA_NAME | varchar(64)  | YES  |     | NULL    |       |\n| TABLE_NAME  | varchar(64)  | YES  |     | NULL    |       |\n+-------------+--------------+------+-----+---------+-------+\n\n\nDESC performance_schema.rpd_tables;\n+-----------------------------+-------------------------------+------+-----+---------+-------+\n| Field                       | Type                          | Null | Key | Default | Extra |\n+-----------------------------+-------------------------------+------+-----+---------+-------+\n| ID                          | int unsigned                  | NO   |     | NULL    |       |\n| SNAPSHOT_SCN                | bigint unsigned               | YES  |     | NULL    |       |\n| PERSISTED_SCN               | bigint unsigned               | YES  |     | NULL    |       |\n| POOL_TYPE                   | varchar(64)                   | YES  |     | NULL    |       |\n| DATA_PLACEMENT_TYPE         | varchar(64)                   | YES  |     | NULL    |       |\n| NROWS                       | bigint unsigned               | NO   |     | NULL    |       |\n| LOAD_STATUS                 | varchar(64)                   | YES  |     | NULL    |       |\n| LOAD_PROGRESS               | double                        | YES  |     | NULL    |       |\n| SIZE_BYTES                  | bigint unsigned               | NO   |     | NULL    |       |\n| TRANSFORMATION_BYTES        | bigint unsigned               | YES  |     | NULL    |       |\n| QUERY_COUNT                 | bigint unsigned               | NO   |     | NULL    |       |\n| LAST_QUERIED                | timestamp(6)                  | YES  |     | NULL    |       |\n| LOAD_START_TIMESTAMP        | timestamp(6)                  | YES  |     | NULL    |       |\n| LOAD_END_TIMESTAMP          | timestamp(6)                  | YES  |     | NULL    |       |\n| RECOVERY_SOURCE             | enum('MySQL','ObjectStorage') | YES  |     | NULL    |       |\n| RECOVERY_START_TIMESTAMP    | timestamp(6)                  | YES  |     | NULL    |       |\n| RECOVERY_END_TIMESTAMP      | timestamp(6)                  | YES  |     | NULL    |       |\n| LOAD_TYPE                   | enum('SELF','USER')           | YES  |     | NULL    |       |\n| LOGICAL_PARTS_LOADED_AT_SCN | longtext                      | YES  |     | NULL    |       |\n+-----------------------------+-------------------------------+------+-----+---------+-------+<\/code><\/pre>\n\n\n\n<p>Now you can run your {analytics} SELECT queries like usually. \ud83d\ude42<\/p>\n\n\n\n<p><a href=\"https:\/\/www.oracle.com\/heatwave\/performance-benchmarks\/\" target=\"_blank\" rel=\"noopener\" title=\"HeatWave MySQL Performance\">More details about HeatWave MySQL Performance<\/a>.<\/p>\n\n\n\n<div style=\"height:50px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h2 class=\"wp-block-heading\">Simplify the process with stored procedure<\/h2>\n\n\n\n<p>The workflow can be easily simplified using stored procedures (<em>since we\u2019re using HeatWave MySQL 8.4, we can\u2019t take advantage of <a href=\"https:\/\/dev.mysql.com\/doc\/refman\/9.2\/en\/stored-routines-js.html\" target=\"_blank\" rel=\"noopener\" title=\"JavaScript Stored Programs\">JavaScript stored procedures<\/a>, which are only available starting from MySQL 9.2.<\/em>). <\/p>\n\n\n\n<p>Let&rsquo;s review the 4 steps:<\/p>\n\n\n\n<p>1. <strong>Clone table<\/strong> (Copy structure without partitions,&#8230; or data)<br>CALL <em><code>clone_table<\/code><\/em>(<em>&lt;schema_name>, &lt;partitioned_table>, &lt;archived_table><\/em>);<br><br>2. <strong>Exchanging Partitions<\/strong><br>CALL <em><code>exchange_partition<\/code><\/em>(<em>&lt;schema_name>, &lt;partitioned_table>, &lt;partition_name>, &lt;archived_table><\/em>);<br><br>3. <strong>Add a primary key<\/strong> <br>CALL <em><code>add_primary_key_column<\/code><\/em>(<em>&lt;schema_name>, &lt;archived_table>, &lt;primary_key><\/em>);<br><br>4. <strong>Load the data into the HeatWave Cluster<\/strong> <br>CALL <strong>sys.heatwave_load<\/strong>(JSON_ARRAY(<em>&lt;schema_name><\/em>), JSON_OBJECT(&lsquo;include_list&rsquo;, JSON_ARRAY(<em>&lt;schema_name>.&lt;archived_table><\/em>)));<\/p>\n\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p>While a robust production solution would be more complex, we can illustrate the core concepts with a basic implementation (<strong>these examples should only be used for testing and learning<\/strong>) : <\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Clone table<\/h3>\n\n\n\n<pre title=\"Clone table\" class=\"wp-block-code\"><code lang=\"sql\" class=\"language-sql\">DELIMITER $$\n\nCREATE PROCEDURE `clone_table`(\n    IN _source_schema VARCHAR(64),\n    IN _source_table VARCHAR(64),\n    IN _destination_table VARCHAR(64)\n) \nBEGIN\n    -- Build the ALTER TABLE statement in order to create the table with the data\n    SET @create_table_stmt = CONCAT(\n            ' CREATE TABLE `',_source_schema, '`.`', _destination_table, \n            '` SELECT * FROM `',_source_schema, '`.`',_source_table, '`', \n            ' WHERE NULL '\n        );\n\n    -- Execute\n    PREPARE stmt FROM @create_table_stmt;\n    EXECUTE stmt;\n    DEALLOCATE PREPARE stmt;\nEND$$\n\nDELIMITER ;<\/code><\/pre>\n\n\n\n<p>Usage:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"sql\" class=\"language-sql\">CALL clone_table('temp_archiving', 'min_temp', 'min_temp_1982');<\/code><\/pre>\n\n\n\n<div style=\"height:25px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h3 class=\"wp-block-heading\">Exchange Partition<\/h3>\n\n\n\n<pre title=\"Exchanging Partitions\" class=\"wp-block-code\"><code lang=\"sql\" class=\"language-sql\">DELIMITER $$\n\nCREATE PROCEDURE exchange_partition(\n    IN _source_schema VARCHAR(64),\n    IN _partitioned_table VARCHAR(64),\n    IN _partition_to_exchange VARCHAR(64),\n    IN _new_table_name VARCHAR(64)\n)\nBEGIN\n    -- Build the ALTER TABLE statement in order to exchange the partition with the newly created table\n    SET @exchange_stmt = CONCAT(\n        ' ALTER TABLE `',_source_schema, '`.`', _partitioned_table, '` ',\n        ' EXCHANGE PARTITION `', _partition_to_exchange, '` ',\n        ' WITH TABLE `',_source_schema, '`.`', _new_table_name, '`'\n    );\n\n    -- Execute\n    PREPARE stmt FROM @exchange_stmt;\n    EXECUTE stmt;\n    DEALLOCATE PREPARE stmt;\nEND$$\n\nDELIMITER ;<\/code><\/pre>\n\n\n\n<p>Usage:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"sql\" class=\"language-sql\">CALL exchange_partition('temp_archiving', 'min_temp', 'p1982', 'min_temp_1982');<\/code><\/pre>\n\n\n\n<div style=\"height:25px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h3 class=\"wp-block-heading\">Add a Primary Key<\/h3>\n\n\n\n<pre title=\"Add a primary key \" class=\"wp-block-code\"><code lang=\"sql\" class=\"language-sql\">DELIMITER $$\n\nCREATE PROCEDURE add_primary_key_column(\n    IN _source_schema VARCHAR(64),\n    IN _table_name VARCHAR(64),\n    IN _column_with_PK VARCHAR(64)\n)\nBEGIN\n    -- Build the ALTER TABLE statement in order to add the PK\n    SET @alter_table_stmt = CONCAT(\n        ' ALTER TABLE `',_source_schema, '`.`', _table_name, '` ',\n        ' ADD COLUMN `', _column_with_PK, '` INT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY FIRST'\n    );\n\n    -- Execute\n    PREPARE stmt FROM @alter_table_stmt;\n    EXECUTE stmt;\n    DEALLOCATE PREPARE stmt;\nEND$$\n\nDELIMITER ;<\/code><\/pre>\n\n\n\n<p>Usage:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"SQL\" class=\"language-SQL\">CALL add_primary_key_column('temp_archiving', 'min_temp_1982', 'id_min_temp_1982');<\/code><\/pre>\n\n\n\n<div style=\"height:50px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p>Finally, we can create a wrapper to perform all 4 steps at once:<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Data Archiving Wrapper<\/h3>\n\n\n\n<pre title=\"Data archiving\" class=\"wp-block-code\"><code lang=\"sql\" class=\"language-sql\">DELIMITER $$\n\nCREATE PROCEDURE `archiving_wrapper`(\n    IN _source_schema VARCHAR(64),\n    IN _source_table VARCHAR(64),\n    IN _destination_table VARCHAR(64),\n    IN _partition_to_exchange VARCHAR(64),\n    IN _column_with_PK VARCHAR(64)\n) \nBEGIN\n    -- Copy structure without indexes, partitions,... and data\n    CALL clone_table(_source_schema, _source_table, _destination_table);\n\n    -- Exchange partition\n    CALL exchange_partition(_source_schema, _source_table, _partition_to_exchange, _destination_table);\n\n    -- Add a primary key \n    CALL add_primary_key_column(_source_schema, _destination_table, _column_with_PK);\n\n    -- Load data into HeatWave Cluster \n    CALL sys.heatwave_load(\n        JSON_ARRAY(_source_schema),\n        JSON_OBJECT(\n            'include_list',\n            JSON_ARRAY(CONCAT(_source_schema, '.', _destination_table))\n        )\n    );             \n\nEND$$\n\nDELIMITER ;<\/code><\/pre>\n\n\n\n<p>Usage:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"sql\" class=\"language-sql\">SQL>\nCALL archiving_wrapper('temp_archiving', 'min_temp', 'min_temp_1982', 'p1982', 'id_min_temp_1982');\n\n\n+------------------------------------------+\n| INITIALIZING HEATWAVE AUTO PARALLEL LOAD |\n+------------------------------------------+\n| Version: 3.21                            |\n|                                          |\n| Load Mode: normal                        |\n| Load Policy: disable_unsupported_columns |\n| Output Mode: normal                      |\n|                                          |\n+------------------------------------------+\n6 rows in set (0.1021 sec)\n\n+------------------------------------------------------------------------+\n| OFFLOAD ANALYSIS                                                       |\n+------------------------------------------------------------------------+\n| Verifying input schemas: 1                                             |\n| User excluded items: 0                                                 |\n|                                                                        |\n| SCHEMA                       OFFLOADABLE    OFFLOADABLE     SUMMARY OF |\n| NAME                              TABLES        COLUMNS     ISSUES     |\n| ------                       -----------    -----------     ---------- |\n| `temp_archiving`                       1              3                |\n|                                                                        |\n| Total offloadable schemas: 1                                           |\n|                                                                        |\n+------------------------------------------------------------------------+\n10 rows in set (0.1021 sec)\n\n+-----------------------------------------------------------------------------------------------------------------------------+\n| CAPACITY ESTIMATION                                                                                                         |\n+-----------------------------------------------------------------------------------------------------------------------------+\n| Default encoding for string columns: VARLEN (unless specified in the schema)                                                |\n| Estimating memory footprint for 1 schema(s)                                                                                 |\n|                                                                                                                             |\n|                                TOTAL       ESTIMATED       ESTIMATED       TOTAL     DICTIONARY      VARLEN       ESTIMATED |\n| SCHEMA                   OFFLOADABLE   HEATWAVE NODE      MYSQL NODE      STRING        ENCODED     ENCODED            LOAD |\n| NAME                          TABLES       FOOTPRINT       FOOTPRINT     COLUMNS        COLUMNS     COLUMNS            TIME |\n| ------                   -----------       ---------       ---------     -------     ----------     -------       --------- |\n| `temp_archiving`                   1        3.27 MiB      256.00 KiB           0              0           0          1.00 s |\n|                                                                                                                             |\n| Sufficient MySQL host memory available to load all tables.                                                                  |\n| Sufficient HeatWave cluster memory available to load all tables.                                                            |\n|                                                                                                                             |\n+-----------------------------------------------------------------------------------------------------------------------------+\n12 rows in set (0.1021 sec)\n\n+---------------------------------------------------------------------------------------------------------------------------------------+\n| EXECUTING LOAD SCRIPT                                                                                                                 |\n+---------------------------------------------------------------------------------------------------------------------------------------+\n| HeatWave Load script generated                                                                                                        |\n|   Retrieve load script containing 3 generated DDL command(s) using the query below:                                                   |\n| Deprecation Notice: \"heatwave_load_report\" will be deprecated, please switch to \"heatwave_autopilot_report\"                           |\n|   SELECT log->>\"$.sql\" AS \"Load Script\" FROM sys.heatwave_autopilot_report WHERE type = \"sql\" ORDER BY id;                            |\n|                                                                                                                                       |\n| Adjusting load parallelism dynamically per internal\/external table.                                                                   |\n| Using current parallelism of 4 thread(s) as maximum for internal tables.                                                              |\n|                                                                                                                                       |\n| Warning: Executing the generated script may alter column definitions and secondary engine flags in the schema                         |\n|                                                                                                                                       |\n| Using SQL_MODE: ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTION |\n|                                                                                                                                       |\n| Proceeding to load 1 table(s) into HeatWave.                                                                                          |\n|                                                                                                                                       |\n| Applying changes will take approximately 113.00 ms                                                                                    |\n|                                                                                                                                       |\n+---------------------------------------------------------------------------------------------------------------------------------------+\n16 rows in set (0.1021 sec)\n\n+--------------------------------------------------+\n| TABLE LOAD                                       |\n+--------------------------------------------------+\n| TABLE (1 of 1): `temp_archiving`.`min_temp_1982` |\n| Commands executed successfully: 3 of 3           |\n| Warnings encountered: 0                          |\n| Table load succeeded!                            |\n|   Total columns loaded: 3                        |\n|   Table loaded using 1 thread(s)                 |\n|   Elapsed time: 2.66 s                           |\n|                                                  |\n+--------------------------------------------------+\n8 rows in set (0.1021 sec)\n\n+-------------------------------------------------------------------------------+\n| LOAD SUMMARY                                                                  |\n+-------------------------------------------------------------------------------+\n|                                                                               |\n| SCHEMA                          TABLES       TABLES      COLUMNS         LOAD |\n| NAME                            LOADED       FAILED       LOADED     DURATION |\n| ------                          ------       ------      -------     -------- |\n| `temp_archiving`                     1            0            3       2.66 s |\n|                                                                               |\n+-------------------------------------------------------------------------------+\n6 rows in set (0.1021 sec)\n\nQuery OK, 0 rows affected (0.1021 sec)<\/code><\/pre>\n\n\n\n<div style=\"height:25px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p>Let&rsquo;s check:<\/p>\n\n\n\n<pre title=\"Was 1982 archived correctly?\" class=\"wp-block-code\"><code lang=\"sql\" class=\"language-sql\">SQL> \nSELECT * FROM min_temp_1982 LIMIT 3;\n+------------------+------------+------+\n| id_min_temp_1982 | d          | temp |\n+------------------+------------+------+\n|                1 | 1982-01-01 | 17.0 |\n|                2 | 1982-01-02 | 15.0 |\n|                3 | 1982-01-03 | 13.5 |\n+------------------+------------+------+\n\n\nSELECT YEAR(d) AS temp_year FROM min_temp_1982 GROUP BY temp_year;\n+-----------+\n| temp_year |\n+-----------+\n|      1982 |\n+-----------+<\/code><\/pre>\n\n\n\n<p>The year 1982 has been correctly archived.<\/p>\n\n\n\n<div style=\"height:25px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<pre title=\"Has data been moved out of the partitioned table?\" class=\"wp-block-code\"><code lang=\"sql\" class=\"language-sql\">SQL> \nSELECT * FROM min_temp LIMIT 3;\n+------------+------+\n| d          | temp |\n+------------+------+\n| 1983-01-01 | 18.4 |\n| 1983-01-02 | 15.0 |\n| 1983-01-03 | 10.9 |\n+------------+------+\n\n\nSELECT YEAR(d) AS temp_year FROM min_temp GROUP BY temp_year ;\n+-----------+\n| temp_year |\n+-----------+\n|      1983 |\n|      1984 |\n|      1985 |\n|      1986 |\n|      1987 |\n|      1988 |\n|      1989 |\n|      1990 |\n+-----------+\n8 rows in set (0.0013 sec)\n\n\nSELECT PARTITION_NAME, PARTITION_ORDINAL_POSITION, PARTITION_DESCRIPTION, TABLE_ROWS \nFROM INFORMATION_SCHEMA.PARTITIONS \nWHERE TABLE_NAME = 'min_temp';\n+----------------+----------------------------+-----------------------+------------+\n| PARTITION_NAME | PARTITION_ORDINAL_POSITION | PARTITION_DESCRIPTION | TABLE_ROWS |\n+----------------+----------------------------+-----------------------+------------+\n| p1981          |                          1 | 1982                  |          0 |\n| p1982          |                          2 | 1983                  |          0 |\n| p1983          |                          3 | 1984                  |        365 |\n| p1984          |                          4 | 1985                  |        365 |\n| p1985          |                          5 | 1986                  |        365 |\n| p1986          |                          6 | 1987                  |        365 |\n| p1987          |                          7 | 1988                  |        365 |\n| p1988          |                          8 | 1989                  |        365 |\n| p1989          |                          9 | 1990                  |        365 |\n| p1990          |                         10 | 1991                  |        365 |\n+----------------+----------------------------+-----------------------+------------+<\/code><\/pre>\n\n\n\n<p>Data was moved out of the partitioned table.<\/p>\n\n\n\n<div style=\"height:25px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<pre title=\"Has the data been offloaded to the HeatWave cluster?\" class=\"wp-block-code\"><code lang=\"sql\" class=\"language-sql\">SQL> \nSELECT name, load_progress, load_status, query_count \nFROM performance_schema.rpd_tables JOIN performance_schema.rpd_table_id USING(id) \nWHERE name LIKE 'temp_archiving%' \nORDER BY name;\n+------------------------------+---------------+---------------------+-------------+\n| name                         | load_progress | load_status         | query_count |\n+------------------------------+---------------+---------------------+-------------+\n| temp_archiving.min_temp_1981 |           100 | AVAIL_RPDGSTABSTATE |           0 |\n| temp_archiving.min_temp_1982 |           100 | AVAIL_RPDGSTABSTATE |           0 |\n+------------------------------+---------------+---------------------+-------------+<\/code><\/pre>\n\n\n\n<p>Data has been offloaded to the HeatWave Cluster.<\/p>\n\n\n\n<p>In our case, this workload would only run once a year after the catch-up phase. But in practice, you could schedule this kind of workflow to run more regularly \u2014 whether with cron (or the <a href=\"https:\/\/dev.mysql.com\/doc\/refman\/8.4\/en\/event-scheduler.html\" target=\"_blank\" rel=\"noopener\" title=\"Using the MySQL Event Scheduler\">event scheduler<\/a> if you are a player \ud83d\ude42 , your favorite open source data orchestration tool like Airflow, Prefect, Dagster,&#8230; or a cloud native managed service like  <a href=\"https:\/\/www.oracle.com\/integration\/data-integration\/#rc30p2\" target=\"_blank\" rel=\"noopener\" title=\"Oracle Cloud Infrastructure Data Integration\">OCI Data Integration<\/a>.<\/p>\n\n\n\n<div style=\"height:50px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h2 class=\"wp-block-heading\">Peroraison<\/h2>\n\n\n\n<p>Archiving data no longer has to mean sacrificing accessibility or performance. With HeatWave MySQL, DBAs and developers can seamlessly move older partitions into regular tables, load them into the HeatWave cluster, and continue running queries at scale \u2014 all with the familiar MySQL syntax they already know.<\/p>\n\n\n\n<p>By combining efficient data archiving with in-memory acceleration, HeatWave allows organizations to strike the balance between keeping production databases lean and still being able to analyze years of historical data instantly. What was once a trade-off between storage efficiency and query performance is now a streamlined workflow that can be automated and adapted to your needs.<\/p>\n\n\n\n<p><strong>With HeatWave, your archived data is no longer just stored; it&rsquo;s ready to deliver actionable insights at the speed of thought<\/strong>.<\/p>\n\n\n\n<div style=\"height:50px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p>Stay tuned for more insights!<\/p>\n\n\n<div class=\"wp-block-image is-style-default\">\n<figure data-wp-context=\"{&quot;imageId&quot;:&quot;69e752e0a3baf&quot;}\" data-wp-interactive=\"core\/image\" data-wp-key=\"69e752e0a3baf\" class=\"aligncenter size-thumbnail wp-lightbox-container\"><img loading=\"lazy\" decoding=\"async\" width=\"830\" height=\"271\" data-wp-class--hide=\"state.isContentHidden\" data-wp-class--show=\"state.isContentVisible\" data-wp-init=\"callbacks.setButtonStyles\" data-wp-on--click=\"actions.showLightbox\" data-wp-on--load=\"callbacks.setButtonStyles\" data-wp-on-window--resize=\"callbacks.setButtonStyles\" src=\"https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2024\/12\/HeatWave_logo.png?resize=150%2C150&amp;ssl=1\" alt=\"HeatWave\" class=\"wp-image-7222\" srcset=\"https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2024\/12\/HeatWave_logo.png?w=830&amp;ssl=1 830w, https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2024\/12\/HeatWave_logo.png?resize=300%2C98&amp;ssl=1 300w, https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2024\/12\/HeatWave_logo.png?resize=800%2C261&amp;ssl=1 800w, https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2024\/12\/HeatWave_logo.png?resize=768%2C251&amp;ssl=1 768w\" sizes=\"auto, (max-width: 830px) 100vw, 830px\" \/><button\n\t\t\tclass=\"lightbox-trigger\"\n\t\t\ttype=\"button\"\n\t\t\taria-haspopup=\"dialog\"\n\t\t\taria-label=\"Agrandir\"\n\t\t\tdata-wp-init=\"callbacks.initTriggerButton\"\n\t\t\tdata-wp-on--click=\"actions.showLightbox\"\n\t\t\tdata-wp-style--right=\"state.imageButtonRight\"\n\t\t\tdata-wp-style--top=\"state.imageButtonTop\"\n\t\t>\n\t\t\t<svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"12\" height=\"12\" fill=\"none\" viewBox=\"0 0 12 12\">\n\t\t\t\t<path fill=\"#fff\" d=\"M2 0a2 2 0 0 0-2 2v2h1.5V2a.5.5 0 0 1 .5-.5h2V0H2Zm2 10.5H2a.5.5 0 0 1-.5-.5V8H0v2a2 2 0 0 0 2 2h2v-1.5ZM8 12v-1.5h2a.5.5 0 0 0 .5-.5V8H12v2a2 2 0 0 1-2 2H8Zm2-12a2 2 0 0 1 2 2v2h-1.5V2a.5.5 0 0 0-.5-.5H8V0h2Z\" \/>\n\t\t\t<\/svg>\n\t\t<\/button><\/figure>\n<\/div>\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p><a href=\"https:\/\/www.linkedin.com\/groups\/12524512\/\" target=\"_blank\" rel=\"noopener\" title=\"Olivier DASINI on Linkedin\">Follow me on Linkedin<\/a><\/p>\n\n\n\n<p>Watch my videos on my <a href=\"https:\/\/www.youtube.com\/channel\/UC12TulyJsJZHoCmby3Nm3WQ\" target=\"_blank\" rel=\"noreferrer noopener\" title=\"Olivier's MySQL Channel\">YouTube channel<\/a> and <a href=\"https:\/\/www.youtube.com\/channel\/UC12TulyJsJZHoCmby3Nm3WQ\/?sub_confirmation=1\" target=\"_blank\" rel=\"noreferrer noopener\" title=\"Subscribe\">subscribe<\/a>.<\/p>\n\n\n\n<p>My <a href=\"https:\/\/www.slideshare.net\/freshdaz\" target=\"_blank\" rel=\"noreferrer noopener\" title=\"Olivier DASINI on Slideshare\">Slideshare account<\/a>.<\/p>\n\n\n\n<p>My <a href=\"https:\/\/speakerdeck.com\/freshdaz\/\" target=\"_blank\" rel=\"noreferrer noopener\" title=\"Olivier DASINI on Speaker Deck\">Speaker Deck account<\/a>.<\/p>\n\n\n\n<div style=\"height:25px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p class=\"has-vivid-red-color has-text-color\"><strong>Thanks for using HeatWave &amp; MySQL!<\/strong><\/p>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>As databases grow, DBAs and developers often face a tough choice:<br \/>\n + Keep historical data in production tables and risk slow queries<br \/>\n + Archive it and lose fast access<\/p>\n<p>With HeatWave, you don\u2019t have to choose. You can archive old partitions, keep recent data in production, and still run lightning-fast queries across all your data.<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"footnotes":""},"categories":[1713,1694,1719,203,339],"tags":[1754,306,1697,202,349],"class_list":["post-8116","post","type-post","status-publish","format-standard","hentry","category-cloud-en","category-heatwave-en","category-mds-en","category-mysql-en","category-tuto-en","tag-archive","tag-cloud","tag-heatwave-fr-en","tag-performance","tag-tuto-en"],"aioseo_notices":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p9LfWW-26U","jetpack-related-posts":[{"id":8393,"url":"https:\/\/dasini.net\/blog\/2025\/10\/07\/let-your-ai-dba-assistant-write-your-mysql-queries\/","url_meta":{"origin":8116,"position":0},"title":"Let Your AI DBA Assistant Write Your MySQL Queries","author":"Olivier DASINI","date":"7 octobre 2025","format":false,"excerpt":"Having explored the innovative MySQL HeatWave technology that converts Natural Language into SQL (Ask Your Database Anything: Natural Language to SQL in MySQL HeatWave), our next article in this series, dives into a practical use case demonstrating how an AI DBA Assistant can significantly simplify your query generation workflow. In\u2026","rel":"","context":"Dans &quot;AI&quot;","block_context":{"text":"AI","link":"https:\/\/dasini.net\/blog\/category\/ai\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2025\/10\/reltime_monitor.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2025\/10\/reltime_monitor.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2025\/10\/reltime_monitor.png?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2025\/10\/reltime_monitor.png?resize=700%2C400&ssl=1 2x"},"classes":[]},{"id":4462,"url":"https:\/\/dasini.net\/blog\/2021\/04\/13\/heatwave-a-mysql-cloud-feature-to-speed-up-your-queries\/","url_meta":{"origin":8116,"position":1},"title":"HeatWave &#8211; A MySQL cloud feature to speed up your queries","author":"Olivier DASINI","date":"13 avril 2021","format":false,"excerpt":"If you have (too) long running select queries it is probably because of lack of relevant indexes, problematic schema that lead to poor queries or inadequate hardware. That said, sometime even if you doing it right, the query execution time could be too long regarding of what the application or\u2026","rel":"","context":"Dans &quot;Cloud&quot;","block_context":{"text":"Cloud","link":"https:\/\/dasini.net\/blog\/category\/cloud\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2021\/04\/MySQL-Database-Service-HeatWave-Replication.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2021\/04\/MySQL-Database-Service-HeatWave-Replication.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2021\/04\/MySQL-Database-Service-HeatWave-Replication.png?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2021\/04\/MySQL-Database-Service-HeatWave-Replication.png?resize=700%2C400&ssl=1 2x"},"classes":[]},{"id":5762,"url":"https:\/\/dasini.net\/blog\/2022\/06\/09\/explore-visualize-your-mysql-heatwave-data-with-superset\/","url_meta":{"origin":8116,"position":2},"title":"Explore &#038; visualize your MySQL HeatWave data with Superset","author":"Olivier DASINI","date":"9 juin 2022","format":false,"excerpt":"In this article I will show you how to properly configure Apache Superset in order to take advantage of a high performance, in-memory query accelerator: MySQL HeatWave.","rel":"","context":"Dans &quot;MySQL&quot;","block_context":{"text":"MySQL","link":"https:\/\/dasini.net\/blog\/category\/mysql-en\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2022\/06\/Configure-Heatwave-Cluster.png?resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2022\/06\/Configure-Heatwave-Cluster.png?resize=350%2C200 1x, https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2022\/06\/Configure-Heatwave-Cluster.png?resize=525%2C300 1.5x, https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2022\/06\/Configure-Heatwave-Cluster.png?resize=700%2C400 2x, https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2022\/06\/Configure-Heatwave-Cluster.png?resize=1050%2C600 3x"},"classes":[]},{"id":6061,"url":"https:\/\/dasini.net\/blog\/2022\/07\/18\/iris-data-set-with-mysql-heatwave-machine-learning-zeppelin\/","url_meta":{"origin":8116,"position":3},"title":"Iris Data Set with MySQL HeatWave Machine Learning  &#038; Zeppelin","author":"Olivier DASINI","date":"18 juillet 2022","format":false,"excerpt":"During the last 20 years MySQL has democratized the usage of transactional databases. Now with MySQL HeatWave we are in the process of democratizing Analytics and Machine Learning. With MySQL HeatWave, valorizing your data has never been so easy! In this article I will show you how easy it is\u2026","rel":"","context":"Dans &quot;MySQL&quot;","block_context":{"text":"MySQL","link":"https:\/\/dasini.net\/blog\/category\/mysql-en\/"},"img":{"alt_text":"MySQL HeatWave a MySQL in-Memory Query Accelerator","src":"https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2022\/06\/HW_ML.png?fit=1012%2C431&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2022\/06\/HW_ML.png?fit=1012%2C431&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2022\/06\/HW_ML.png?fit=1012%2C431&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2022\/06\/HW_ML.png?fit=1012%2C431&ssl=1&resize=700%2C400 2x"},"classes":[]},{"id":5915,"url":"https:\/\/dasini.net\/blog\/2022\/07\/05\/interactively-explore-visualize-your-mysql-heatwave-data-with-apache-zeppelin\/","url_meta":{"origin":8116,"position":4},"title":"Interactively explore &#038; visualize your MySQL HeatWave data with Apache Zeppelin","author":"Olivier DASINI","date":"5 juillet 2022","format":false,"excerpt":"In this article I will show you how to properly configure Apache Zeppelin in order to take advantage of the Analytics and Machine Learning capabilities of MySQL HeatWave the MySQL in-memory query accelerator.","rel":"","context":"Dans &quot;Machine Learning&quot;","block_context":{"text":"Machine Learning","link":"https:\/\/dasini.net\/blog\/category\/machine-learning-en\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2022\/06\/Configure-Heatwave-Cluster.png?resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2022\/06\/Configure-Heatwave-Cluster.png?resize=350%2C200 1x, https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2022\/06\/Configure-Heatwave-Cluster.png?resize=525%2C300 1.5x, https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2022\/06\/Configure-Heatwave-Cluster.png?resize=700%2C400 2x, https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2022\/06\/Configure-Heatwave-Cluster.png?resize=1050%2C600 3x"},"classes":[]},{"id":4662,"url":"https:\/\/dasini.net\/blog\/2021\/08\/03\/discovering-mysql-database-service-episode-1-introduction\/","url_meta":{"origin":8116,"position":5},"title":"Discovering MySQL Database Service &#8211; Episode 1 &#8211; Introduction","author":"Olivier DASINI","date":"3 ao\u00fbt 2021","format":false,"excerpt":"This is the first episode of \u201cDiscovering MySQL Database Service\u201c, a series of tutorials where I will show you, step by step, how to use MySQL Database Service and some other Oracle Cloud Infrastructure services. Like any series, in this episode I\u2019m going to give you some context and set\u2026","rel":"","context":"Dans &quot;Cloud&quot;","block_context":{"text":"Cloud","link":"https:\/\/dasini.net\/blog\/category\/cloud\/"},"img":{"alt_text":"MySQL Database Service","src":"https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2021\/06\/MDS_car_801x600.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2021\/06\/MDS_car_801x600.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2021\/06\/MDS_car_801x600.png?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/dasini.net\/blog\/wp-content\/uploads\/2021\/06\/MDS_car_801x600.png?resize=700%2C400&ssl=1 2x"},"classes":[]}],"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/dasini.net\/blog\/wp-json\/wp\/v2\/posts\/8116","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dasini.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dasini.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dasini.net\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/dasini.net\/blog\/wp-json\/wp\/v2\/comments?post=8116"}],"version-history":[{"count":164,"href":"https:\/\/dasini.net\/blog\/wp-json\/wp\/v2\/posts\/8116\/revisions"}],"predecessor-version":[{"id":8337,"href":"https:\/\/dasini.net\/blog\/wp-json\/wp\/v2\/posts\/8116\/revisions\/8337"}],"wp:attachment":[{"href":"https:\/\/dasini.net\/blog\/wp-json\/wp\/v2\/media?parent=8116"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dasini.net\/blog\/wp-json\/wp\/v2\/categories?post=8116"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dasini.net\/blog\/wp-json\/wp\/v2\/tags?post=8116"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}