-
-
Save redstreet/6f1addb87c667826fb79b509d5d88a51 to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
├── documents | |
│ ├── Assets | |
│ │ ├── Banks | |
│ │ │ ├── Bank-of-USA <--- *.pdf, *.jpg, etc. (documents) | |
│ ├── Expenses | |
├── ingest | |
| ├── ingest_output | |
│ ├── filingcabinet <--- bean-file's root directory | |
│ │ ├── Assets | |
│ │ │ ├── Banks | |
│ │ │ │ ├── Bank-of-USA <--- *.ofx, *.csv, *.xlsx, etc. | |
│ │ │ ├── ... | |
│ │ ├── Income | |
│ │ │ └── Employment | |
│ │ └── Liabilities | |
│ │ └── Credit-Cards | |
│ │ └── ... | |
├── prices <--- prices.bc | |
├── main.bc <--- include "source/Assets.Banks.xyz.bc", etc. | |
├── source <--- Assets.Banks.[...].bc, ... |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/zsh | |
setopt extendedglob | |
# exit when any command fails | |
# set -e | |
# keep track of the last executed command | |
trap 'last_command=$current_command; current_command=$BASH_COMMAND' DEBUG | |
# echo an error message before exiting | |
trap 'echo "\"${last_command}\" command filed with exit code $?."' EXIT | |
# https://stackoverflow.com/questions/192249/how-do-i-parse-command-line-arguments-in-bash | |
while [[ "$#" -gt 0 ]]; do | |
case $1 in | |
-x|--unimplemented) target="$2"; shift ;; # just for documentation | |
-n|--nofile) nofile=1 ;; | |
*) echo "Unknown parameter passed: $1"; exit 1 ;; | |
esac | |
shift | |
done | |
INGEST_ROOT=$BEAN_ROOT/ingest | |
INGEST_OUTPUT=$INGEST_ROOT/ingest_output | |
ROOT_DIR=~/Downloads/ | |
SCRIPT_DIR=$(dirname -- "$0") | |
alias bi="bean-identify ${INGEST_ROOT}/my.import " | |
alias be="bean-extract -o --dir ${INGEST_OUTPUT} ${INGEST_ROOT}/my.import " | |
alias bf="bean-file -o ${INGEST_ROOT}/filingcabinet ${INGEST_ROOT}/my.import " | |
files=("${(@f)$(find $ROOT_DIR -maxdepth 1 -iname "*.qfx" -o -iname "*.ofx" -o -iname "*.xml" -o -iname "*.csv" -o -iname "*.xlsx")}") | |
echo "Ingesting ${#files[@]} files." | |
rm -frv ${INGEST_OUTPUT} | |
mkdir -p ${INGEST_OUTPUT} | |
# So zerosum doesn't run: both for performance and correctness (smart_importer) | |
if be -f <(echo 'plugin "beancount.plugins.auto_accounts"'; cat ${INGEST_ROOT}/../source/* ) $files ; then | |
echo "Return value of bean-extract: $?" | |
if [[ "$nofile"x != "1x" ]]; then | |
bf $files | |
fi | |
fi | |
echo "-----------------------------------" | |
echo "${#files[@]} files processed. Creating prices.db." | |
# Optional: Collect all price entries in a separate prices.bc file | |
touch ${INGEST_OUTPUT}/noaccount.bc | |
grep price ${INGEST_OUTPUT}/noaccount.bc >> ${BEAN_ROOT}/prices/prices.bc || echo '' | |
echo 'price' >> ${INGEST_OUTPUT}/noaccount.bc | |
echo '' >> ${INGEST_OUTPUT}/noaccount.bc | |
sed -i '/price/d' ${INGEST_OUTPUT}/noaccount.bc | |
sed -i '/^$/d' ${INGEST_OUTPUT}/noaccount.bc | |
[[ ! -s ${INGEST_OUTPUT}/noaccount.bc ]] && rm -fv ${INGEST_OUTPUT}/noaccount.bc | |
echo "Import done. Editing all imported files." | |
vim -p ${INGEST_OUTPUT}/**/*.bc | |
echo "Append to master? (Ctrl+c to exit)" | |
read | |
${INGEST_ROOT}/append_to_master.sh | |
rmdir --ignore-fail-on-non-empty ~/Downloads/skipped |
ingest_output is a sibling of filingcabinet. I've updated the gist to illusrate this, and main.bc
-o
and --dir
need a patch to be applied. I have the patch in another gist. Search for patch in this article.
Hope that helps!
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I've been a bit confused about
ingest_output
. Is this a directory that would be in directory-layout next to filingcabinet? Is this where the main.beancount would be?The
-o --dir
options of bean-extract are erroring out for me so I've been trying something likebe -f $files > $(INGEST_OUTPUT)/test.beancount