This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import bauplan | |
| import boto3 | |
| from datetime import datetime | |
| def main(): | |
| try: | |
| # Initialize clients | |
| bpln = bauplan.Client() | |
| username = bpln.info().user.username | |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| from typing import List, Optional, Set | |
| class Column: | |
| def __init__(self, name: str): | |
| self.name = name | |
| def __repr__(self): | |
| return f"Column(name={self.name})" | |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| """ | |
| Standalone script to write the same dataframe in different storage formats and then | |
| measure the time it takes to read it into the user process. | |
| Setup: | |
| * create a virtualenv with pyarrow==17.0.0, boto3==1.35.7, posix-ipc==1.1.1 | |
| * make sure you have an S3 bucket and appropriate AWS credentials to write / read from it |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| SELECT | |
| c_count, | |
| count(*) AS custdist | |
| FROM ( | |
| SELECT | |
| c_custkey, | |
| count(o_orderkey) | |
| FROM | |
| read_parquet('s3://bucket/customer.parquet') AS customer | |
| LEFT OUTER JOIN |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| from reclist.datasets import CoveoDataset | |
| from reclist.recommenders.prod2vec import CoveoP2VRecModel | |
| from reclist.reclist import CoveoCartRecList | |
| if __name__ == "__main__": | |
| # get the coveo data challenge dataset as a RecDataset object | |
| coveo_dataset = CoveoDataset() | |
| # re-use a skip-gram model from reclist to train a latent product space, to be used |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| class MyRecList(RecList): | |
| @rec_test(test_type='stats') | |
| def basic_stats(self): | |
| """ | |
| Basic statistics on training, test and prediction data | |
| """ | |
| from reclist.metrics.standard_metrics import statistics | |
| return statistics(self._x_train, | |
| self._y_train, |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| // max expression length (i.e. how many atomic facts can be considered) | |
| var MAX_EXP_LENGTH = 5; | |
| var TARGET_EXP = { | |
| 'q': 1000000000, | |
| 'd': 'target', | |
| 'u': '$' | |
| } | |
| var topK = 100; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| // specify how much data to use | |
| var N = 25; // you can try 25, 50, 100 | |
| var dataType = 'exp'; // linear, constant, exp | |
| ///fold: load observed data | |
| print("Using " + N + " data points for data type: " + dataType); | |
| var dataString = { | |
| 'exp': '[{"x": 8.528104691935328, "y": 369.4385603078311}, {"x": 5.800314416734446, "y": 56.17361673332166}, {"x": 6.957475968211479, "y": 124.0710506473754}, {"x": 9.481786398402916, "y": 715.0458977557804}, {"x": 8.735115980299934, "y": 426.23403384951274}, {"x": 3.045444240247178, "y": 8.356747238145411}, {"x": 6.9001768350511785, "y": 119.71324924833438}, {"x": 4.697285583404605, "y": 25.034180007597616}, {"x": 4.793562296412884, "y": 27.708924544629504}, {"x": 5.821197003876745, "y": 56.65940024103008}, {"x": 5.288087142321756, "y": 38.57248324353289}, {"x": 7.908547013925951, "y": 241.1127051063566}, {"x": 6.522075450293987, "y": 91.98603466343532}, {"x": 5.243350032985656, "y": 38.66132447348228}, {"x": 5.887726465490851, "y": 58.81299495311647}, {"x": 5. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| var unaryOps = ['log', 'round', 'sqrt']; | |
| var binaryOps = ['add', 'sub', 'mul', 'div']; | |
| var constants = ['x', 'z', 'u', 'q']; | |
| var randomNumber = function() { | |
| return [uniformDraw(_.range(10))]; | |
| } | |
| var randomConstant = function() { | |
| return [uniformDraw(constants)]; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| var drawLines = function(drawObj, lines, strokeW){ | |
| var line = lines[0]; | |
| drawObj.line(line[0], line[1], line[2], line[3], strokeW); | |
| if (lines.length > 1) { | |
| drawLines(drawObj, lines.slice(1), strokeW); | |
| } | |
| } | |
| var drawSamples = function(sam){ | |
| drawLines(Draw(50, 50, true), sam); |
NewerOlder