Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save albinkjellin/3ca92cef023ff2701e62c529cbd158a6 to your computer and use it in GitHub Desktop.
Save albinkjellin/3ca92cef023ff2701e62c529cbd158a6 to your computer and use it in GitHub Desktop.
Product Table Scan console
(2.1.0b2) $soda scan warehouse.yml tables/product.yml
| 2.1.0b2
| Scanning tables/product.yml ...
| Soda cloud: cloud.soda.io
| Soda Cloud scan start
| > /api/command (login with API key credentials)
| < 200 (login ok, token received)
| Executing SQL query:
SELECT column_name, data_type, is_nullable
FROM information_schema.columns
WHERE lower(table_name) = 'product'
AND table_catalog = 'sodademo1'
AND table_schema = 'public'
| SQL took 0:00:00.013264
| 9 columns:
| productid (character varying)
| productname (character varying)
| productcategory (character varying)
| size (integer)
| weight (numeric)
| purchaseprice (numeric)
| sellingprice (numeric)
| amountinstock (integer)
| manufacturer (character varying)
| Query measurement: schema = [{'name': 'productid', 'type': 'character varying'}, {'name': 'productname', 'type': 'character varying'}, {'name': 'productcategory', 'type': 'character varying'}, {'name': 'size', 'type': 'integer'}, {'name': 'weight', 'type': 'numeric'}, {'name': 'purchaseprice', 'type': 'numeric'}, {'name': 'sellingprice', 'type': 'numeric'}, {'name': 'amountinstock', 'type': 'integer'}, {'name': 'manufacturer', 'type': 'character varying'}]
| Soda Cloud scan send measurements
| Executing SQL query:
SELECT
COUNT(*),
COUNT(CASE WHEN NOT ("productid" IS NULL) THEN 1 END),
COUNT(CASE WHEN NOT ("productid" IS NULL) AND ("productid" ~* '^[0-9a-fA-F]{8}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{12}$') THEN 1 END),
AVG(CASE WHEN NOT ("productid" IS NULL) AND ("productid" ~* '^[0-9a-fA-F]{8}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{12}$') THEN LENGTH("productid") END),
MIN(CASE WHEN NOT ("productid" IS NULL) AND ("productid" ~* '^[0-9a-fA-F]{8}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{12}$') THEN LENGTH("productid") END),
MAX(CASE WHEN NOT ("productid" IS NULL) AND ("productid" ~* '^[0-9a-fA-F]{8}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{12}$') THEN LENGTH("productid") END),
COUNT(CASE WHEN NOT ("productname" IS NULL) THEN 1 END),
COUNT(CASE WHEN NOT ("productname" IS NULL) THEN 1 END),
AVG(CASE WHEN NOT ("productname" IS NULL) THEN LENGTH("productname") END),
MIN(CASE WHEN NOT ("productname" IS NULL) THEN LENGTH("productname") END),
MAX(CASE WHEN NOT ("productname" IS NULL) THEN LENGTH("productname") END),
COUNT(CASE WHEN NOT ("productcategory" IS NULL) THEN 1 END),
COUNT(CASE WHEN NOT ("productcategory" IS NULL) THEN 1 END),
AVG(CASE WHEN NOT ("productcategory" IS NULL) THEN LENGTH("productcategory") END),
MIN(CASE WHEN NOT ("productcategory" IS NULL) THEN LENGTH("productcategory") END),
MAX(CASE WHEN NOT ("productcategory" IS NULL) THEN LENGTH("productcategory") END),
COUNT(CASE WHEN NOT ("size" IS NULL) THEN 1 END),
COUNT(CASE WHEN NOT ("size" IS NULL) AND ("size" <= 95) THEN 1 END),
MIN(CASE WHEN NOT ("size" IS NULL) AND ("size" <= 95) THEN "size" END),
MAX(CASE WHEN NOT ("size" IS NULL) AND ("size" <= 95) THEN "size" END),
AVG(CASE WHEN NOT ("size" IS NULL) AND ("size" <= 95) THEN "size" END),
SUM(CASE WHEN NOT ("size" IS NULL) AND ("size" <= 95) THEN "size" END),
VARIANCE(CASE WHEN NOT ("size" IS NULL) AND ("size" <= 95) THEN "size" END),
STDDEV(CASE WHEN NOT ("size" IS NULL) AND ("size" <= 95) THEN "size" END),
COUNT(CASE WHEN NOT ("weight" IS NULL) THEN 1 END),
COUNT(CASE WHEN NOT ("weight" IS NULL) THEN 1 END),
MIN("weight"),
MAX("weight"),
AVG("weight"),
SUM("weight"),
VARIANCE("weight"),
STDDEV("weight"),
COUNT(CASE WHEN NOT ("purchaseprice" IS NULL) THEN 1 END),
COUNT(CASE WHEN NOT ("purchaseprice" IS NULL) THEN 1 END),
MIN("purchaseprice"),
MAX("purchaseprice"),
AVG("purchaseprice"),
SUM("purchaseprice"),
VARIANCE("purchaseprice"),
STDDEV("purchaseprice"),
COUNT(CASE WHEN NOT ("sellingprice" IS NULL) THEN 1 END),
COUNT(CASE WHEN NOT ("sellingprice" IS NULL) THEN 1 END),
MIN("sellingprice"),
MAX("sellingprice"),
AVG("sellingprice"),
SUM("sellingprice"),
VARIANCE("sellingprice"),
STDDEV("sellingprice"),
COUNT(CASE WHEN NOT ("amountinstock" IS NULL) THEN 1 END),
COUNT(CASE WHEN NOT ("amountinstock" IS NULL) THEN 1 END),
MIN("amountinstock"),
MAX("amountinstock"),
AVG("amountinstock"),
SUM("amountinstock"),
VARIANCE("amountinstock"),
STDDEV("amountinstock"),
COUNT(CASE WHEN NOT ("manufacturer" IS NULL) THEN 1 END),
COUNT(CASE WHEN NOT ("manufacturer" IS NULL) THEN 1 END),
AVG(CASE WHEN NOT ("manufacturer" IS NULL) THEN LENGTH("manufacturer") END),
MIN(CASE WHEN NOT ("manufacturer" IS NULL) THEN LENGTH("manufacturer") END),
MAX(CASE WHEN NOT ("manufacturer" IS NULL) THEN LENGTH("manufacturer") END)
FROM "public"."product"
| SQL took 0:00:00.020660
| Query measurement: row_count = 279
| Query measurement: values_count(productid) = 279
| Query measurement: valid_count(productid) = 279
| Query measurement: avg_length(productid) = 36.0000000000000000
| Query measurement: min_length(productid) = 36
| Query measurement: max_length(productid) = 36
| Query measurement: values_count(productname) = 279
| Query measurement: valid_count(productname) = 279
| Query measurement: avg_length(productname) = 13.4050179211469534
| Query measurement: min_length(productname) = 3
| Query measurement: max_length(productname) = 28
| Query measurement: values_count(productcategory) = 265
| Query measurement: valid_count(productcategory) = 265
| Query measurement: avg_length(productcategory) = 6.5471698113207547
| Query measurement: min_length(productcategory) = 4
| Query measurement: max_length(productcategory) = 11
| Query measurement: values_count(size) = 279
| Query measurement: valid_count(size) = 268
| Query measurement: min(size) = 1
| Query measurement: max(size) = 95
| Query measurement: avg(size) = 48.7761194029850746
| Query measurement: sum(size) = 13072
| Query measurement: variance(size) = 695.0583039856895299
| Query measurement: stddev(size) = 26.3639584278554333
| Query measurement: values_count(weight) = 279
| Query measurement: valid_count(weight) = 279
| Query measurement: min(weight) = 2.2
| Query measurement: max(weight) = 49.98
| Query measurement: avg(weight) = 26.3918279569892473
| Query measurement: sum(weight) = 7363.32
| Query measurement: variance(weight) = 192.3688574379206312
| Query measurement: stddev(weight) = 13.8697100704348046
| Query measurement: values_count(purchaseprice) = 279
| Query measurement: valid_count(purchaseprice) = 279
| Query measurement: min(purchaseprice) = 21.48
| Query measurement: max(purchaseprice) = 598.46
| Query measurement: avg(purchaseprice) = 314.1169534050179211
| Query measurement: sum(purchaseprice) = 87638.63
| Query measurement: variance(purchaseprice) = 28566.881048598541
| Query measurement: stddev(purchaseprice) = 169.017398656465
| Query measurement: values_count(sellingprice) = 279
| Query measurement: valid_count(sellingprice) = 279
| Query measurement: min(sellingprice) = 16.37
| Query measurement: max(sellingprice) = 718.15
| Query measurement: avg(sellingprice) = 371.9779211469534050
| Query measurement: sum(sellingprice) = 103781.84
| Query measurement: variance(sellingprice) = 42228.218057533328
| Query measurement: stddev(sellingprice) = 205.495056041583
| Query measurement: values_count(amountinstock) = 279
| Query measurement: valid_count(amountinstock) = 279
| Query measurement: min(amountinstock) = 0
| Query measurement: max(amountinstock) = 199
| Query measurement: avg(amountinstock) = 49.8530465949820789
| Query measurement: sum(amountinstock) = 13909
| Query measurement: variance(amountinstock) = 3894.4711327712023929
| Query measurement: stddev(amountinstock) = 62.4056979191099376
| Query measurement: values_count(manufacturer) = 279
| Query measurement: valid_count(manufacturer) = 279
| Query measurement: avg_length(manufacturer) = 16.4336917562724014
| Query measurement: min_length(manufacturer) = 7
| Query measurement: max_length(manufacturer) = 31
| Derived measurement: missing_percentage(productid) = 0.0
| Derived measurement: missing_count(productid) = 0
| Derived measurement: values_percentage(productid) = 100.0
| Derived measurement: invalid_percentage(productid) = 0.0
| Derived measurement: invalid_count(productid) = 0
| Derived measurement: valid_percentage(productid) = 100.0
| Derived measurement: missing_percentage(productname) = 0.0
| Derived measurement: missing_count(productname) = 0
| Derived measurement: values_percentage(productname) = 100.0
| Derived measurement: invalid_percentage(productname) = 0.0
| Derived measurement: invalid_count(productname) = 0
| Derived measurement: valid_percentage(productname) = 100.0
| Derived measurement: missing_percentage(productcategory) = 5.017921146953405
| Derived measurement: missing_count(productcategory) = 14
| Derived measurement: values_percentage(productcategory) = 94.9820788530466
| Derived measurement: invalid_percentage(productcategory) = 0.0
| Derived measurement: invalid_count(productcategory) = 0
| Derived measurement: valid_percentage(productcategory) = 94.9820788530466
| Derived measurement: missing_percentage(size) = 0.0
| Derived measurement: missing_count(size) = 0
| Derived measurement: values_percentage(size) = 100.0
| Derived measurement: invalid_percentage(size) = 3.942652329749104
| Derived measurement: invalid_count(size) = 11
| Derived measurement: valid_percentage(size) = 96.05734767025089
| Derived measurement: missing_percentage(weight) = 0.0
| Derived measurement: missing_count(weight) = 0
| Derived measurement: values_percentage(weight) = 100.0
| Derived measurement: invalid_percentage(weight) = 0.0
| Derived measurement: invalid_count(weight) = 0
| Derived measurement: valid_percentage(weight) = 100.0
| Derived measurement: missing_percentage(purchaseprice) = 0.0
| Derived measurement: missing_count(purchaseprice) = 0
| Derived measurement: values_percentage(purchaseprice) = 100.0
| Derived measurement: invalid_percentage(purchaseprice) = 0.0
| Derived measurement: invalid_count(purchaseprice) = 0
| Derived measurement: valid_percentage(purchaseprice) = 100.0
| Derived measurement: missing_percentage(sellingprice) = 0.0
| Derived measurement: missing_count(sellingprice) = 0
| Derived measurement: values_percentage(sellingprice) = 100.0
| Derived measurement: invalid_percentage(sellingprice) = 0.0
| Derived measurement: invalid_count(sellingprice) = 0
| Derived measurement: valid_percentage(sellingprice) = 100.0
| Derived measurement: missing_percentage(amountinstock) = 0.0
| Derived measurement: missing_count(amountinstock) = 0
| Derived measurement: values_percentage(amountinstock) = 100.0
| Derived measurement: invalid_percentage(amountinstock) = 0.0
| Derived measurement: invalid_count(amountinstock) = 0
| Derived measurement: valid_percentage(amountinstock) = 100.0
| Derived measurement: missing_percentage(manufacturer) = 0.0
| Derived measurement: missing_count(manufacturer) = 0
| Derived measurement: values_percentage(manufacturer) = 100.0
| Derived measurement: invalid_percentage(manufacturer) = 0.0
| Derived measurement: invalid_count(manufacturer) = 0
| Derived measurement: valid_percentage(manufacturer) = 100.0
| Soda Cloud scan send measurements
| Executing SQL query:
WITH group_by_value AS (
SELECT
"productid" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("productid" IS NULL) AND ("productid" ~* '^[0-9a-fA-F]{8}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{12}$')
GROUP BY "productid"
)
SELECT COUNT(*),
COUNT(CASE WHEN frequency = 1 THEN 1 END),
SUM(frequency)
FROM group_by_value
| SQL took 0:00:00.004485
| Query measurement: distinct(productid) = 279
| Query measurement: unique_count(productid) = 279
| Derived measurement: duplicate_count(productid) = 0
| Derived measurement: uniqueness(productid) = 100
| Executing SQL query:
WITH group_by_value AS (
SELECT
"productid" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("productid" IS NULL) AND ("productid" ~* '^[0-9a-fA-F]{8}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{12}$')
GROUP BY "productid"
)
SELECT value
FROM group_by_value
ORDER BY value ASC
LIMIT 5
| SQL took 0:00:00.003252
| Query measurement: mins(productid) = ['01bd328a-a1aa-11eb-ac84-1e00da0c4374', '01bd3a3c-a1aa-11eb-ac84-1e00da0c4374', '01bd3e7e-a1aa-11eb-ac84-1e00da0c4374', '01bd42a2-a1aa-11eb-ac84-1e00da0c4374', '01bd4536-a1aa-11eb-ac84-1e00da0c4374']
| Executing SQL query:
WITH group_by_value AS (
SELECT
"productid" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("productid" IS NULL) AND ("productid" ~* '^[0-9a-fA-F]{8}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{12}$')
GROUP BY "productid"
)
SELECT value
FROM group_by_value
ORDER BY value DESC
LIMIT 5
| SQL took 0:00:00.004184
| Query measurement: maxs(productid) = ['f3a4a930-a1a9-11eb-b0cf-1e00da0c4374', 'f3a4a6ba-a1a9-11eb-b0cf-1e00da0c4374', 'f3a4a2f0-a1a9-11eb-b0cf-1e00da0c4374', 'f3a49f26-a1a9-11eb-b0cf-1e00da0c4374', 'f3a499f4-a1a9-11eb-b0cf-1e00da0c4374']
| Executing SQL query:
WITH group_by_value AS (
SELECT
"productid" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("productid" IS NULL) AND ("productid" ~* '^[0-9a-fA-F]{8}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{12}$')
GROUP BY "productid"
)
SELECT value, frequency
FROM group_by_value
ORDER BY frequency DESC
LIMIT 5
| SQL took 0:00:00.004347
| Query measurement: frequent_values(productid) = [{'value': '01bd8b9a-a1aa-11eb-ac84-1e00da0c4374', 'frequency': 1}, {'value': '01bda594-a1aa-11eb-ac84-1e00da0c4374', 'frequency': 1}, {'value': '01bd770e-a1aa-11eb-ac84-1e00da0c4374', 'frequency': 1}, {'value': 'f3a456d8-a1a9-11eb-b0cf-1e00da0c4374', 'frequency': 1}, {'value': '01bd5ee0-a1aa-11eb-ac84-1e00da0c4374', 'frequency': 1}]
| Soda Cloud scan send measurements
| Executing SQL query:
WITH group_by_value AS (
SELECT
"productname" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("productname" IS NULL)
GROUP BY "productname"
)
SELECT COUNT(*),
COUNT(CASE WHEN frequency = 1 THEN 1 END),
SUM(frequency)
FROM group_by_value
| SQL took 0:00:00.001083
| Query measurement: distinct(productname) = 210
| Query measurement: unique_count(productname) = 177
| Derived measurement: duplicate_count(productname) = 33
| Derived measurement: uniqueness(productname) = 75.17985611510791366906474820
| Executing SQL query:
WITH group_by_value AS (
SELECT
"productname" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("productname" IS NULL)
GROUP BY "productname"
)
SELECT value
FROM group_by_value
ORDER BY value ASC
LIMIT 5
| SQL took 0:00:00.000416
| Query measurement: mins(productname) = ['Awesome Chair', 'Awesome Cotton Ball', 'Awesome Mouse', 'Awesome Soft Salad', 'Awesome Wooden Mouse']
| Executing SQL query:
WITH group_by_value AS (
SELECT
"productname" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("productname" IS NULL)
GROUP BY "productname"
)
SELECT value
FROM group_by_value
ORDER BY value DESC
LIMIT 5
| SQL took 0:00:00.000367
| Query measurement: maxs(productname) = ['Wooden Towels', 'Wooden Chips', 'Wooden Cheese', 'Wooden Bacon', 'Used Steel Ball']
| Executing SQL query:
WITH group_by_value AS (
SELECT
"productname" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("productname" IS NULL)
GROUP BY "productname"
)
SELECT value, frequency
FROM group_by_value
ORDER BY frequency DESC
LIMIT 5
| SQL took 0:00:00.000325
| Query measurement: frequent_values(productname) = [{'value': 'Electric Guitar', 'frequency': 14}, {'value': 'Chips', 'frequency': 7}, {'value': 'Sausages', 'frequency': 7}, {'value': 'Chair', 'frequency': 6}, {'value': 'Gloves', 'frequency': 5}]
| Soda Cloud scan send measurements
| Executing SQL query:
WITH group_by_value AS (
SELECT
"productcategory" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("productcategory" IS NULL)
GROUP BY "productcategory"
)
SELECT COUNT(*),
COUNT(CASE WHEN frequency = 1 THEN 1 END),
SUM(frequency)
FROM group_by_value
| SQL took 0:00:00.000724
| Query measurement: distinct(productcategory) = 22
| Query measurement: unique_count(productcategory) = 0
| Derived measurement: duplicate_count(productcategory) = 22
| Derived measurement: uniqueness(productcategory) = 7.954545454545454545454545455
| Executing SQL query:
WITH group_by_value AS (
SELECT
"productcategory" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("productcategory" IS NULL)
GROUP BY "productcategory"
)
SELECT value
FROM group_by_value
ORDER BY value ASC
LIMIT 5
| SQL took 0:00:00.000306
| Query measurement: mins(productcategory) = ['Automotive', 'Baby', 'Beauty', 'Books', 'Clothing']
| Executing SQL query:
WITH group_by_value AS (
SELECT
"productcategory" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("productcategory" IS NULL)
GROUP BY "productcategory"
)
SELECT value
FROM group_by_value
ORDER BY value DESC
LIMIT 5
| SQL took 0:00:00.000243
| Query measurement: maxs(productcategory) = ['Toys', 'Tools', 'Sports', 'Shoes', 'Outdoors']
| Executing SQL query:
WITH group_by_value AS (
SELECT
"productcategory" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("productcategory" IS NULL)
GROUP BY "productcategory"
)
SELECT value, frequency
FROM group_by_value
ORDER BY frequency DESC
LIMIT 5
| SQL took 0:00:00.000311
| Query measurement: frequent_values(productcategory) = [{'value': 'Grocery', 'frequency': 20}, {'value': 'Home', 'frequency': 17}, {'value': 'Beauty', 'frequency': 17}, {'value': 'Industrial', 'frequency': 17}, {'value': 'Health', 'frequency': 15}]
| Soda Cloud scan send measurements
| Executing SQL query:
WITH group_by_value AS (
SELECT
"size" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("size" IS NULL) AND ("size" <= 95)
GROUP BY "size"
)
SELECT COUNT(*),
COUNT(CASE WHEN frequency = 1 THEN 1 END),
SUM(frequency)
FROM group_by_value
| SQL took 0:00:00.001357
| Query measurement: distinct(size) = 90
| Query measurement: unique_count(size) = 20
| Derived measurement: duplicate_count(size) = 70
| Derived measurement: uniqueness(size) = 33.33333333333333333333333333
| Executing SQL query:
WITH group_by_value AS (
SELECT
"size" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("size" IS NULL) AND ("size" <= 95)
GROUP BY "size"
)
SELECT value
FROM group_by_value
ORDER BY value ASC
LIMIT 5
| SQL took 0:00:00.000403
| Query measurement: mins(size) = [1, 2, 3, 4, 5]
| Executing SQL query:
WITH group_by_value AS (
SELECT
"size" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("size" IS NULL) AND ("size" <= 95)
GROUP BY "size"
)
SELECT value
FROM group_by_value
ORDER BY value DESC
LIMIT 5
| SQL took 0:00:00.000486
| Query measurement: maxs(size) = [95, 94, 93, 91, 90]
| Executing SQL query:
WITH group_by_value AS (
SELECT
"size" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("size" IS NULL) AND ("size" <= 95)
GROUP BY "size"
)
SELECT value, frequency
FROM group_by_value
ORDER BY frequency DESC
LIMIT 5
| SQL took 0:00:00.000423
| Query measurement: frequent_values(size) = [{'value': 47, 'frequency': 7}, {'value': 34, 'frequency': 7}, {'value': 65, 'frequency': 7}, {'value': 88, 'frequency': 6}, {'value': 80, 'frequency': 6}]
| Soda Cloud scan send measurements
| Executing SQL query:
WITH group_by_value AS (
SELECT
"weight" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("weight" IS NULL)
GROUP BY "weight"
)
SELECT COUNT(*),
COUNT(CASE WHEN frequency = 1 THEN 1 END),
SUM(frequency)
FROM group_by_value
| SQL took 0:00:00.001601
| Query measurement: distinct(weight) = 269
| Query measurement: unique_count(weight) = 259
| Derived measurement: duplicate_count(weight) = 10
| Derived measurement: uniqueness(weight) = 96.40287769784172661870503597
| Executing SQL query:
WITH group_by_value AS (
SELECT
"weight" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("weight" IS NULL)
GROUP BY "weight"
)
SELECT value
FROM group_by_value
ORDER BY value ASC
LIMIT 5
| SQL took 0:00:00.000567
| Query measurement: mins(weight) = [Decimal('2.2'), Decimal('2.35'), Decimal('2.57'), Decimal('2.69'), Decimal('2.81')]
| Executing SQL query:
WITH group_by_value AS (
SELECT
"weight" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("weight" IS NULL)
GROUP BY "weight"
)
SELECT value
FROM group_by_value
ORDER BY value DESC
LIMIT 5
| SQL took 0:00:00.000513
| Query measurement: maxs(weight) = [Decimal('49.98'), Decimal('49.96'), Decimal('49.85'), Decimal('49.46'), Decimal('49.4')]
| Executing SQL query:
WITH group_by_value AS (
SELECT
"weight" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("weight" IS NULL)
GROUP BY "weight"
)
SELECT value, frequency
FROM group_by_value
ORDER BY frequency DESC
LIMIT 5
| SQL took 0:00:00.000522
| Query measurement: frequent_values(weight) = [{'value': Decimal('39.14'), 'frequency': 2}, {'value': Decimal('8.15'), 'frequency': 2}, {'value': Decimal('12.83'), 'frequency': 2}, {'value': Decimal('19.62'), 'frequency': 2}, {'value': Decimal('43.5'), 'frequency': 2}]
| Soda Cloud scan send measurements
| Executing SQL query:
WITH group_by_value AS (
SELECT
"purchaseprice" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("purchaseprice" IS NULL)
GROUP BY "purchaseprice"
)
SELECT COUNT(*),
COUNT(CASE WHEN frequency = 1 THEN 1 END),
SUM(frequency)
FROM group_by_value
| SQL took 0:00:00.001036
| Query measurement: distinct(purchaseprice) = 276
| Query measurement: unique_count(purchaseprice) = 273
| Derived measurement: duplicate_count(purchaseprice) = 3
| Derived measurement: uniqueness(purchaseprice) = 98.92086330935251798561151079
| Executing SQL query:
WITH group_by_value AS (
SELECT
"purchaseprice" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("purchaseprice" IS NULL)
GROUP BY "purchaseprice"
)
SELECT value
FROM group_by_value
ORDER BY value ASC
LIMIT 5
| SQL took 0:00:00.000466
| Query measurement: mins(purchaseprice) = [Decimal('21.48'), Decimal('21.93'), Decimal('23.31'), Decimal('24.61'), Decimal('27.48')]
| Executing SQL query:
WITH group_by_value AS (
SELECT
"purchaseprice" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("purchaseprice" IS NULL)
GROUP BY "purchaseprice"
)
SELECT value
FROM group_by_value
ORDER BY value DESC
LIMIT 5
| SQL took 0:00:00.000414
| Query measurement: maxs(purchaseprice) = [Decimal('598.46'), Decimal('598.17'), Decimal('596.25'), Decimal('589.8'), Decimal('589.74')]
| Executing SQL query:
WITH group_by_value AS (
SELECT
"purchaseprice" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("purchaseprice" IS NULL)
GROUP BY "purchaseprice"
)
SELECT value, frequency
FROM group_by_value
ORDER BY frequency DESC
LIMIT 5
| SQL took 0:00:00.000406
| Query measurement: frequent_values(purchaseprice) = [{'value': Decimal('89.24'), 'frequency': 2}, {'value': Decimal('589.3'), 'frequency': 2}, {'value': Decimal('549.1'), 'frequency': 2}, {'value': Decimal('436.13'), 'frequency': 1}, {'value': Decimal('532.51'), 'frequency': 1}]
| Soda Cloud scan send measurements
| Executing SQL query:
WITH group_by_value AS (
SELECT
"sellingprice" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("sellingprice" IS NULL)
GROUP BY "sellingprice"
)
SELECT COUNT(*),
COUNT(CASE WHEN frequency = 1 THEN 1 END),
SUM(frequency)
FROM group_by_value
| SQL took 0:00:00.001109
| Query measurement: distinct(sellingprice) = 276
| Query measurement: unique_count(sellingprice) = 273
| Derived measurement: duplicate_count(sellingprice) = 3
| Derived measurement: uniqueness(sellingprice) = 98.92086330935251798561151079
| Executing SQL query:
WITH group_by_value AS (
SELECT
"sellingprice" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("sellingprice" IS NULL)
GROUP BY "sellingprice"
)
SELECT value
FROM group_by_value
ORDER BY value ASC
LIMIT 5
| SQL took 0:00:00.000556
| Query measurement: mins(sellingprice) = [Decimal('16.37'), Decimal('19.17'), Decimal('25.78'), Decimal('26.32'), Decimal('27.97')]
| Executing SQL query:
WITH group_by_value AS (
SELECT
"sellingprice" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("sellingprice" IS NULL)
GROUP BY "sellingprice"
)
SELECT value
FROM group_by_value
ORDER BY value DESC
LIMIT 5
| SQL took 0:00:00.000451
| Query measurement: maxs(sellingprice) = [Decimal('718.15'), Decimal('717.80'), Decimal('715.50'), Decimal('707.76'), Decimal('707.69')]
| Executing SQL query:
WITH group_by_value AS (
SELECT
"sellingprice" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("sellingprice" IS NULL)
GROUP BY "sellingprice"
)
SELECT value, frequency
FROM group_by_value
ORDER BY frequency DESC
LIMIT 5
| SQL took 0:00:00.000445
| Query measurement: frequent_values(sellingprice) = [{'value': Decimal('707.16'), 'frequency': 2}, {'value': Decimal('107.09'), 'frequency': 2}, {'value': Decimal('658.92'), 'frequency': 2}, {'value': Decimal('555.58'), 'frequency': 1}, {'value': Decimal('230.11'), 'frequency': 1}]
| Soda Cloud scan send measurements
| Executing SQL query:
WITH group_by_value AS (
SELECT
"amountinstock" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("amountinstock" IS NULL)
GROUP BY "amountinstock"
)
SELECT COUNT(*),
COUNT(CASE WHEN frequency = 1 THEN 1 END),
SUM(frequency)
FROM group_by_value
| SQL took 0:00:00.000922
| Query measurement: distinct(amountinstock) = 93
| Query measurement: unique_count(amountinstock) = 57
| Derived measurement: duplicate_count(amountinstock) = 36
| Derived measurement: uniqueness(amountinstock) = 33.09352517985611510791366906
| Executing SQL query:
WITH group_by_value AS (
SELECT
"amountinstock" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("amountinstock" IS NULL)
GROUP BY "amountinstock"
)
SELECT value
FROM group_by_value
ORDER BY value ASC
LIMIT 5
| SQL took 0:00:00.000422
| Query measurement: mins(amountinstock) = [0, 1, 2, 3, 4]
| Executing SQL query:
WITH group_by_value AS (
SELECT
"amountinstock" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("amountinstock" IS NULL)
GROUP BY "amountinstock"
)
SELECT value
FROM group_by_value
ORDER BY value DESC
LIMIT 5
| SQL took 0:00:00.000322
| Query measurement: maxs(amountinstock) = [199, 197, 194, 193, 192]
| Executing SQL query:
WITH group_by_value AS (
SELECT
"amountinstock" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("amountinstock" IS NULL)
GROUP BY "amountinstock"
)
SELECT value, frequency
FROM group_by_value
ORDER BY frequency DESC
LIMIT 5
| SQL took 0:00:00.000337
| Query measurement: frequent_values(amountinstock) = [{'value': 12, 'frequency': 18}, {'value': 9, 'frequency': 16}, {'value': 13, 'frequency': 15}, {'value': 2, 'frequency': 15}, {'value': 11, 'frequency': 13}]
| Soda Cloud scan send measurements
| Executing SQL query:
WITH group_by_value AS (
SELECT
"manufacturer" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("manufacturer" IS NULL)
GROUP BY "manufacturer"
)
SELECT COUNT(*),
COUNT(CASE WHEN frequency = 1 THEN 1 END),
SUM(frequency)
FROM group_by_value
| SQL took 0:00:00.001053
| Query measurement: distinct(manufacturer) = 275
| Query measurement: unique_count(manufacturer) = 271
| Derived measurement: duplicate_count(manufacturer) = 4
| Derived measurement: uniqueness(manufacturer) = 98.56115107913669064748201439
| Executing SQL query:
WITH group_by_value AS (
SELECT
"manufacturer" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("manufacturer" IS NULL)
GROUP BY "manufacturer"
)
SELECT value
FROM group_by_value
ORDER BY value ASC
LIMIT 5
| SQL took 0:00:00.000550
| Query measurement: mins(manufacturer) = ['Aguilar-Bryant', 'Alexander, Taylor and Hahn', 'Allen, Perez and Campos', 'Anderson Group', 'Anderson, Kidd and Smith']
| Executing SQL query:
WITH group_by_value AS (
SELECT
"manufacturer" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("manufacturer" IS NULL)
GROUP BY "manufacturer"
)
SELECT value
FROM group_by_value
ORDER BY value DESC
LIMIT 5
| SQL took 0:00:00.000483
| Query measurement: maxs(manufacturer) = ['Young-Rhodes', 'Young, Sims and Wilkinson', 'Young, Roberts and Mills', 'Wright-Esparza', 'Wright, Stanton and Adkins']
| Executing SQL query:
WITH group_by_value AS (
SELECT
"manufacturer" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("manufacturer" IS NULL)
GROUP BY "manufacturer"
)
SELECT value, frequency
FROM group_by_value
ORDER BY frequency DESC
LIMIT 5
| SQL took 0:00:00.000611
| Query measurement: frequent_values(manufacturer) = [{'value': 'Butler Ltd', 'frequency': 2}, {'value': 'Patterson PLC', 'frequency': 2}, {'value': 'Burton and Sons', 'frequency': 2}, {'value': 'Smith-Johnson', 'frequency': 2}, {'value': 'Johnson-Wong', 'frequency': 1}]
| Soda Cloud scan send measurements
| Executing SQL query:
WITH group_by_value AS (
SELECT
"size" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("size" IS NULL) AND ("size" <= 95)
GROUP BY "size"
)
SELECT
SUM(CASE WHEN value < 5.7 THEN frequency END),
SUM(CASE WHEN 5.7 <= value and value < 10.4 THEN frequency END),
SUM(CASE WHEN 10.4 <= value and value < 15.1 THEN frequency END),
SUM(CASE WHEN 15.1 <= value and value < 19.8 THEN frequency END),
SUM(CASE WHEN 19.8 <= value and value < 24.5 THEN frequency END),
SUM(CASE WHEN 24.5 <= value and value < 29.2 THEN frequency END),
SUM(CASE WHEN 29.2 <= value and value < 33.9 THEN frequency END),
SUM(CASE WHEN 33.9 <= value and value < 38.6 THEN frequency END),
SUM(CASE WHEN 38.6 <= value and value < 43.3 THEN frequency END),
SUM(CASE WHEN 43.3 <= value and value < 48.0 THEN frequency END),
SUM(CASE WHEN 48.0 <= value and value < 52.7 THEN frequency END),
SUM(CASE WHEN 52.7 <= value and value < 57.4 THEN frequency END),
SUM(CASE WHEN 57.4 <= value and value < 62.1 THEN frequency END),
SUM(CASE WHEN 62.1 <= value and value < 66.8 THEN frequency END),
SUM(CASE WHEN 66.8 <= value and value < 71.5 THEN frequency END),
SUM(CASE WHEN 71.5 <= value and value < 76.2 THEN frequency END),
SUM(CASE WHEN 76.2 <= value and value < 80.9 THEN frequency END),
SUM(CASE WHEN 80.9 <= value and value < 85.6 THEN frequency END),
SUM(CASE WHEN 85.6 <= value and value < 90.3 THEN frequency END),
SUM(CASE WHEN 90.3 <= value THEN frequency END)
FROM group_by_value
| SQL took 0:00:00.002583
| Query measurement: histogram(size) = {'boundaries': [1.0, 5.7, 10.4, 15.1, 19.8, 24.5, 29.2, 33.9, 38.6, 43.3, 48.0, 52.7, 57.4, 62.1, 66.8, 71.5, 76.2, 80.9, 85.6, 90.3, 95.0], 'frequencies': [11, 9, 14, 9, 18, 11, 17, 21, 11, 12, 15, 14, 10, 14, 13, 15, 15, 13, 19, 7]}
| Soda Cloud scan send measurements
| Executing SQL query:
WITH group_by_value AS (
SELECT
"weight" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("weight" IS NULL)
GROUP BY "weight"
)
SELECT
SUM(CASE WHEN value < 4.589 THEN frequency END),
SUM(CASE WHEN 4.589 <= value and value < 6.978 THEN frequency END),
SUM(CASE WHEN 6.978 <= value and value < 9.367 THEN frequency END),
SUM(CASE WHEN 9.367 <= value and value < 11.756 THEN frequency END),
SUM(CASE WHEN 11.756 <= value and value < 14.145 THEN frequency END),
SUM(CASE WHEN 14.145 <= value and value < 16.534 THEN frequency END),
SUM(CASE WHEN 16.534 <= value and value < 18.923 THEN frequency END),
SUM(CASE WHEN 18.923 <= value and value < 21.312 THEN frequency END),
SUM(CASE WHEN 21.312 <= value and value < 23.701 THEN frequency END),
SUM(CASE WHEN 23.701 <= value and value < 26.09 THEN frequency END),
SUM(CASE WHEN 26.09 <= value and value < 28.479 THEN frequency END),
SUM(CASE WHEN 28.479 <= value and value < 30.868 THEN frequency END),
SUM(CASE WHEN 30.868 <= value and value < 33.257 THEN frequency END),
SUM(CASE WHEN 33.257 <= value and value < 35.646 THEN frequency END),
SUM(CASE WHEN 35.646 <= value and value < 38.035 THEN frequency END),
SUM(CASE WHEN 38.035 <= value and value < 40.424 THEN frequency END),
SUM(CASE WHEN 40.424 <= value and value < 42.813 THEN frequency END),
SUM(CASE WHEN 42.813 <= value and value < 45.202 THEN frequency END),
SUM(CASE WHEN 45.202 <= value and value < 47.591 THEN frequency END),
SUM(CASE WHEN 47.591 <= value THEN frequency END)
FROM group_by_value
| SQL took 0:00:00.001797
| Query measurement: histogram(weight) = {'boundaries': [2.2, 4.589, 6.978, 9.367, 11.756, 14.145, 16.534, 18.923, 21.312, 23.701, 26.09, 28.479, 30.868, 33.257, 35.646, 38.035, 40.424, 42.813, 45.202, 47.591, 49.98], 'frequencies': [17, 10, 13, 16, 13, 11, 8, 17, 25, 11, 11, 12, 18, 14, 7, 21, 15, 10, 13, 17]}
| Soda Cloud scan send measurements
| Executing SQL query:
WITH group_by_value AS (
SELECT
"purchaseprice" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("purchaseprice" IS NULL)
GROUP BY "purchaseprice"
)
SELECT
SUM(CASE WHEN value < 50.329 THEN frequency END),
SUM(CASE WHEN 50.329 <= value and value < 79.178 THEN frequency END),
SUM(CASE WHEN 79.178 <= value and value < 108.027 THEN frequency END),
SUM(CASE WHEN 108.027 <= value and value < 136.876 THEN frequency END),
SUM(CASE WHEN 136.876 <= value and value < 165.725 THEN frequency END),
SUM(CASE WHEN 165.725 <= value and value < 194.574 THEN frequency END),
SUM(CASE WHEN 194.574 <= value and value < 223.423 THEN frequency END),
SUM(CASE WHEN 223.423 <= value and value < 252.272 THEN frequency END),
SUM(CASE WHEN 252.272 <= value and value < 281.121 THEN frequency END),
SUM(CASE WHEN 281.121 <= value and value < 309.97 THEN frequency END),
SUM(CASE WHEN 309.97 <= value and value < 338.819 THEN frequency END),
SUM(CASE WHEN 338.819 <= value and value < 367.668 THEN frequency END),
SUM(CASE WHEN 367.668 <= value and value < 396.517 THEN frequency END),
SUM(CASE WHEN 396.517 <= value and value < 425.366 THEN frequency END),
SUM(CASE WHEN 425.366 <= value and value < 454.215 THEN frequency END),
SUM(CASE WHEN 454.215 <= value and value < 483.064 THEN frequency END),
SUM(CASE WHEN 483.064 <= value and value < 511.913 THEN frequency END),
SUM(CASE WHEN 511.913 <= value and value < 540.762 THEN frequency END),
SUM(CASE WHEN 540.762 <= value and value < 569.611 THEN frequency END),
SUM(CASE WHEN 569.611 <= value THEN frequency END)
FROM group_by_value
| SQL took 0:00:00.002049
| Query measurement: histogram(purchaseprice) = {'boundaries': [21.48, 50.329, 79.178, 108.027, 136.876, 165.725, 194.574, 223.423, 252.272, 281.121, 309.97, 338.819, 367.668, 396.517, 425.366, 454.215, 483.064, 511.913, 540.762, 569.611, 598.46], 'frequencies': [9, 14, 22, 9, 18, 16, 8, 16, 17, 11, 8, 16, 12, 12, 15, 18, 11, 18, 15, 14]}
| Soda Cloud scan send measurements
| Executing SQL query:
WITH group_by_value AS (
SELECT
"sellingprice" AS value,
COUNT(*) AS frequency
FROM "public"."product"
WHERE NOT ("sellingprice" IS NULL)
GROUP BY "sellingprice"
)
SELECT
SUM(CASE WHEN value < 51.459 THEN frequency END),
SUM(CASE WHEN 51.459 <= value and value < 86.548 THEN frequency END),
SUM(CASE WHEN 86.548 <= value and value < 121.637 THEN frequency END),
SUM(CASE WHEN 121.637 <= value and value < 156.726 THEN frequency END),
SUM(CASE WHEN 156.726 <= value and value < 191.815 THEN frequency END),
SUM(CASE WHEN 191.815 <= value and value < 226.904 THEN frequency END),
SUM(CASE WHEN 226.904 <= value and value < 261.993 THEN frequency END),
SUM(CASE WHEN 261.993 <= value and value < 297.082 THEN frequency END),
SUM(CASE WHEN 297.082 <= value and value < 332.171 THEN frequency END),
SUM(CASE WHEN 332.171 <= value and value < 367.26 THEN frequency END),
SUM(CASE WHEN 367.26 <= value and value < 402.349 THEN frequency END),
SUM(CASE WHEN 402.349 <= value and value < 437.438 THEN frequency END),
SUM(CASE WHEN 437.438 <= value and value < 472.527 THEN frequency END),
SUM(CASE WHEN 472.527 <= value and value < 507.616 THEN frequency END),
SUM(CASE WHEN 507.616 <= value and value < 542.705 THEN frequency END),
SUM(CASE WHEN 542.705 <= value and value < 577.794 THEN frequency END),
SUM(CASE WHEN 577.794 <= value and value < 612.883 THEN frequency END),
SUM(CASE WHEN 612.883 <= value and value < 647.972 THEN frequency END),
SUM(CASE WHEN 647.972 <= value and value < 683.061 THEN frequency END),
SUM(CASE WHEN 683.061 <= value THEN frequency END)
FROM group_by_value
| SQL took 0:00:00.002296
| Query measurement: histogram(sellingprice) = {'boundaries': [16.37, 51.459, 86.548, 121.637, 156.726, 191.815, 226.904, 261.993, 297.082, 332.171, 367.26, 402.349, 437.438, 472.527, 507.616, 542.705, 577.794, 612.883, 647.972, 683.061, 718.15], 'frequencies': [10, 13, 22, 10, 16, 11, 17, 13, 12, 17, 8, 15, 12, 13, 14, 18, 10, 19, 15, 14]}
| Soda Cloud scan send measurements
| Soda Cloud scan send measurements
| Sending failed rows for sql metric INVALID_PRICE to Soda Cloud
| Executing SQL query:
select * from product
where purchaseprice > sellingprice
| SQL took 0:00:00.001028
| Query measurement: INVALID_PRICE = 5
| Soda Cloud scan send measurements
| Test sqlmetric(0) INVALID_PRICE(INVALID_PRICE == 0) failed with metric values {"INVALID_PRICE": 5}
| Soda Cloud scan send test results
| Sent failed rows for sql metric (5/5) to Soda Cloud
| Test test(row_count > 0) passed with metric values {"row_count": 279}
| Soda Cloud scan send test results
| Test column(productid) test(invalid_percentage == 0) passed with metric values {"invalid_percentage": 0.0}
| Test column(productcategory) test(missing_count < 10) failed with metric values {"missing_count": 14}
| Test column(size) test(invalid_percentage < 5) passed with metric values {"invalid_percentage": 3.942652329749104}
| Soda Cloud scan send test results
| Sending sample product.sample
| Executing SQL query:
SELECT *
FROM "public"."product"
LIMIT 50;
| SQL took 0:00:00.002082
| Sent sample product.sample (50/279) to Soda Cloud
| Sending sample product.productcategory.missing
| Executing SQL query:
SELECT *
FROM "public"."product"
WHERE "productcategory" IS NULL
LIMIT 50;
| SQL took 0:00:00.001004
| Sent sample product.productcategory.missing (14/14) to Soda Cloud
| Sending sample product.size.invalid
| Executing SQL query:
SELECT *
FROM "public"."product"
WHERE NOT ("size" IS NULL) AND NOT (("size" <= 95))
LIMIT 50;
| SQL took 0:00:00.001066
| Sent sample product.size.invalid (11/11) to Soda Cloud
| Executed 42 queries in 0:00:10.948448
| Soda Cloud scan end ok
| Scan summary ------
| 194 measurements computed
| 5 tests executed
| 2 of 5 tests failed:
| Test sqlmetric(0) INVALID_PRICE(INVALID_PRICE == 0) failed with metric values {"INVALID_PRICE": 5}
| Test column(productcategory) test(missing_count < 10) failed with metric values {"missing_count": 14}
| Exiting with code 1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment