Skip to content

Instantly share code, notes, and snippets.

@masayuki038
Last active March 27, 2020 05:51
Show Gist options
  • Save masayuki038/6820f551f1a6d4e1ec7b3929ffad35ad to your computer and use it in GitHub Desktop.
Save masayuki038/6820f551f1a6d4e1ec7b3929ffad35ad to your computer and use it in GitHub Desktop.

before

cstore_test=# select product_group, count(*) from customer_reviews group by product_group;
 product_group |  count
---------------+---------
 DVD           |  121418
 Video         |  142235
 Music         |  300628
 Book          | 1198218
 Software      |       1
 CE            |       1
 Toy           |       3
(7 rows)

cstore_test=# EXPLAIN ANALYZE VERBOSE select product_group, count(*) from customer_reviews_cstore group by product_group;
                                                               QUERY PLAN
----------------------------------------------------------------------------------------------------------------------------------------
 HashAggregate  (cost=36258.75..36260.75 rows=200 width=40) (actual time=3358.244..3358.246 rows=7 loops=1)
   Output: book.product_group, count(*)
   Group Key: book.product_group
   ->  Append  (cost=0.00..27446.23 rows=1762504 width=32) (actual time=0.242..2392.405 rows=1762504 loops=1)
         ->  Foreign Scan on public.book  (cost=0.00..12676.43 rows=1198218 width=32) (actual time=0.241..590.246 rows=1198218 loops=1)
               Output: book.product_group
               CStore File Size: 68247552
         ->  Foreign Scan on public.ce  (cost=0.00..0.34 rows=1 width=32) (actual time=0.108..0.109 rows=1 loops=1)
               Output: ce.product_group
               CStore File Size: 32768
         ->  Foreign Scan on public.dvd  (cost=0.00..1281.18 rows=121418 width=32) (actual time=0.201..59.203 rows=121418 loops=1)
               Output: dvd.product_group
               CStore File Size: 6586368
         ->  Foreign Scan on public.music  (cost=0.00..3172.28 rows=300628 width=32) (actual time=0.293..155.498 rows=300628 loops=1)
               Output: music.product_group
               CStore File Size: 16318464
         ->  Foreign Scan on public.software  (cost=0.00..0.34 rows=1 width=32) (actual time=0.047..0.048 rows=1 loops=1)
               Output: software.product_group
               CStore File Size: 32768
         ->  Foreign Scan on public.toy  (cost=0.00..0.36 rows=3 width=32) (actual time=0.045..0.048 rows=3 loops=1)
               Output: toy.product_group
               CStore File Size: 32768
         ->  Foreign Scan on public.video  (cost=0.00..1502.77 rows=142235 width=32) (actual time=0.221..69.641 rows=142235 loops=1)
               Output: video.product_group
               CStore File Size: 7905280
 Planning Time: 0.806 ms
 Execution Time: 3358.408 ms
(27 rows)

modify

# git diff
diff --git a/cstore_fdw.c b/cstore_fdw.c
index fd08621..220806e 100644
--- a/cstore_fdw.c
+++ b/cstore_fdw.c
@@ -149,7 +149,8 @@ static TupleTableSlot * CStoreExecForeignInsert(EState *executorState,
                                                                                                TupleTableSlot *tupleSlot,
                                                                                                TupleTableSlot *planSlot);
 static void CStoreEndForeignModify(EState *executorState, ResultRelInfo *relationInfo);
-
+static bool CStoreIsForeignScanParallelSafe(PlannerInfo *root, RelOptInfo *rel,
+                                                                         RangeTblEntry *rte);

 /* declarations for dynamic loading */
 PG_MODULE_MAGIC;
@@ -1191,6 +1192,7 @@ cstore_fdw_handler(PG_FUNCTION_ARGS)
        fdwRoutine->BeginForeignModify = CStoreBeginForeignModify;
        fdwRoutine->ExecForeignInsert = CStoreExecForeignInsert;
        fdwRoutine->EndForeignModify = CStoreEndForeignModify;
+       fdwRoutine->IsForeignScanParallelSafe = CStoreIsForeignScanParallelSafe;

        PG_RETURN_POINTER(fdwRoutine);
 }
@@ -2234,3 +2236,13 @@ CStoreEndForeignModify(EState *executorState, ResultRelInfo *relationInfo)
        }
 }

+/*
+ *  CStoreIsForeignScanParallelSafe
+ */
+static bool
+CStoreIsForeignScanParallelSafe(PlannerInfo *root, RelOptInfo *rel,
+                                                                         RangeTblEntry *rte)
+{
+       return true;
+}
+

after

cstore_test=# select product_group, count(*) from customer_reviews_cstore group by product_group;
 product_group |  count
---------------+---------
 Book          | 1198218
 CE            |       1
 DVD           |  121418
 Music         |  300628
 Software      |       1
 Toy           |       3
 Video         |  142235
(7 rows)

cstore_test=# EXPLAIN ANALYZE VERBOSE select product_group, count(*) from customer_reviews_cstore group by product_group;
                                                                        QUERY PLAN
----------------------------------------------------------------------------------------------------------------------------------------------------------
 Finalize GroupAggregate  (cost=16350.32..16394.32 rows=200 width=40) (actual time=2486.607..2486.629 rows=7 loops=1)
   Output: book.product_group, count(*)
   Group Key: book.product_group
   ->  Gather Merge  (cost=16350.32..16392.32 rows=400 width=40) (actual time=2486.591..2486.603 rows=7 loops=1)
         Output: book.product_group, (PARTIAL count(*))
         Workers Planned: 2
         Workers Launched: 2
         ->  Sort  (cost=16350.32..16350.32 rows=200 width=40) (actual time=1294.222..1294.225 rows=2 loops=3)
               Output: book.product_group, (PARTIAL count(*))
               Sort Key: book.product_group
               Sort Method: quicksort  Memory: 25kB
               Worker 0:  Sort Method: quicksort  Memory: 25kB
               Worker 1:  Sort Method: quicksort  Memory: 25kB
               Worker 0: actual time=2455.745..2455.747 rows=1 loops=1
               Worker 1: actual time=746.120..746.121 rows=1 loops=1
               ->  Partial HashAggregate  (cost=16348.32..16350.32 rows=200 width=40) (actual time=1294.154..1294.157 rows=2 loops=3)
                     Output: book.product_group, PARTIAL count(*)
                     Group Key: book.product_group
                     Worker 0: actual time=2455.653..2455.655 rows=1 loops=1
                     Worker 1: actual time=746.042..746.044 rows=1 loops=1
                     ->  Parallel Append  (cost=0.00..16348.32 rows=734377 width=32) (actual time=1.459..922.325 rows=587501 loops=3)
                           Worker 0: actual time=2.256..1754.035 rows=1198218 loops=1
                           Worker 1: actual time=1.938..529.985 rows=300628 loops=1
                           ->  Foreign Scan on public.book  (cost=0.00..12676.43 rows=1198218 width=32) (actual time=2.253..648.646 rows=1198218 loops=1)
                                 Output: book.product_group
                                 CStore File Size: 68247552
                                 Worker 0: actual time=2.253..648.646 rows=1198218 loops=1
                           ->  Foreign Scan on public.music  (cost=0.00..3172.28 rows=300628 width=32) (actual time=1.935..194.581 rows=300628 loops=1)
                                 Output: music.product_group
                                 CStore File Size: 16318464
                                 Worker 1: actual time=1.935..194.581 rows=300628 loops=1
                           ->  Foreign Scan on public.video  (cost=0.00..1502.77 rows=142235 width=32) (actual time=2.971..97.716 rows=142235 loops=1)
                                 Output: video.product_group
                                 CStore File Size: 7905280
                           ->  Foreign Scan on public.dvd  (cost=0.00..1281.18 rows=121418 width=32) (actual time=0.808..78.930 rows=121418 loops=1)
                                 Output: dvd.product_group
                                 CStore File Size: 6586368
                           ->  Foreign Scan on public.toy  (cost=0.00..0.36 rows=3 width=32) (actual time=0.055..0.057 rows=3 loops=1)
                                 Output: toy.product_group
                                 CStore File Size: 32768
                           ->  Foreign Scan on public.ce  (cost=0.00..0.34 rows=1 width=32) (actual time=0.053..0.054 rows=1 loops=1)
                                 Output: ce.product_group
                                 CStore File Size: 32768
                           ->  Foreign Scan on public.software  (cost=0.00..0.34 rows=1 width=32) (actual time=0.182..0.183 rows=1 loops=1)
                                 Output: software.product_group
                                 CStore File Size: 32768
 Planning Time: 11.811 ms
 Execution Time: 2491.126 ms
(48 rows)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment