diff options
| author | Christian Cleberg <[email protected]> | 2025-11-11 22:49:13 -0600 |
|---|---|---|
| committer | Christian Cleberg <[email protected]> | 2025-11-11 22:49:13 -0600 |
| commit | 51a7a02f0c96d49b68fbcc155414c218207fa270 (patch) | |
| tree | 845af8aad0e8769352efc02fcd1044eed9cc1ec1 /content/blog/2022-03-03-financial-database.org | |
| parent | 7d3e80ebf1dc770eac0e21890b74f18ba2d15a6b (diff) | |
| download | cleberg.net-51a7a02f0c96d49b68fbcc155414c218207fa270.tar.gz cleberg.net-51a7a02f0c96d49b68fbcc155414c218207fa270.tar.bz2 cleberg.net-51a7a02f0c96d49b68fbcc155414c218207fa270.zip | |
fix grammar in 2022 posts
Diffstat (limited to 'content/blog/2022-03-03-financial-database.org')
| -rw-r--r-- | content/blog/2022-03-03-financial-database.org | 159 |
1 files changed, 69 insertions, 90 deletions
diff --git a/content/blog/2022-03-03-financial-database.org b/content/blog/2022-03-03-financial-database.org index 40612b5..b82fccf 100644 --- a/content/blog/2022-03-03-financial-database.org +++ b/content/blog/2022-03-03-financial-database.org @@ -4,60 +4,49 @@ #+slug: financial-database * Personal Financial Tracking -:PROPERTIES: -:CUSTOM_ID: personal-financial-tracking -:END: -For the last 6-ish years, I've tracked my finances in a spreadsheet. -This is common practice in the business world, but any good dev will -cringe at the thought of storing long-term data in a spreadsheet. A -spreadsheet is not for long-term storage or as a source of data to pull -data/reports. - -As I wanted to expand the functionality of my financial data (e.g., -adding more reports), I decided to migrate the data into a database. To -run reports, I would query the database and use a language like Python -or Javascript to process the data, perform calculations, and visualize -the data. + +For the last 6-ish years, I've tracked my finances in a spreadsheet. This is +common practice in the business world, but any good dev will cringe at the +thought of storing long-term data in a spreadsheet. A spreadsheet is not for +long-term storage or as a source of data to pull data/reports. + +As I wanted to expand the functionality of my financial data (e.g., adding more +reports), I decided to migrate the data into a database. To run reports, I would +query the database and use a language like Python or Javascript to process the +data, perform calculations, and visualize the data. * SQLite -:PROPERTIES: -:CUSTOM_ID: sqlite -:END: -When choosing the type of database I wanted to use for this project, I -was split between three options: - -1. MySQL: The database I have the most experience with and have used for - years. + +When choosing the type of database I wanted to use for this project, I was split +between three options: + +1. MySQL: The database I have the most experience with and have used for years. 2. PostgreSQL: A database I'm new to, but want to learn. -3. SQLite: A database that I've used for a couple projects and have - moderate experience. - -I ended up choosing SQLite since it can be maintained within a single -=.sqlite= file, which allows me more flexibility for storage and backup. -I keep this file in my cloud storage and pull it up whenever needed. - -** GUI Editing -:PROPERTIES: -:CUSTOM_ID: gui-editing -:END: -Since I didn't want to try and import 1000--1500 records into my new -database via the command line, I opted to use -[[https://sqlitebrowser.org/][DB Browser for SQLite (DB4S)]] as a GUI -tool. This application is excellent, and I don't see myself going back -to the CLI when working in this database. +3. SQLite: A database that I've used for a couple projects and have moderate + experience. + +I ended up choosing SQLite since it can be maintained within a single =.sqlite= +file, which allows me more flexibility for storage and backup. I keep this file +in my cloud storage and pull it up whenever needed. + +** Visual Editing + +Since I didn't want to try and import 1000--1500 records into my new database +via the command line, I opted to use [[https://sqlitebrowser.org/][DB Browser for SQLite (DB4S)]] as a GUI +(graphical user interface) tool. This application is excellent, and I don't see +myself going back to the CLI (command line interface) when working in this +database. DB4S allows you to copy a range of cells from a spreadsheet and paste it -straight into the SQL table. I used this process for all 36 accounts, -1290 account statements, and 126 pay statements. Overall, I'm guessing -this took anywhere between 4--8 hours. In comparison, it probably took -me 2-3 days to initially create the spreadsheet. +straight into the SQL table. I used this process for all 36 accounts, 1290 +account statements, and 126 pay statements. Overall, I'm guessing this took +anywhere between 4--8 hours. In comparison, it probably took me 2-3 days to +initially create the spreadsheet. ** Schema -:PROPERTIES: -:CUSTOM_ID: schema -:END: -The schema for this database is actually extremely simple and involves -only three tables (for now): + +The schema for this database is actually extremely simple and involves only +three tables (for now): 1. Accounts 2. Statements @@ -65,9 +54,9 @@ only three tables (for now): *Accounts* -The Accounts table contains summary information about an account, such -as a car loan or a credit card. By viewing this table, you can find -high-level data, such as interest rate, credit line, or owner. +The Accounts table contains summary information about an account, such as a car +loan or a credit card. By viewing this table, you can find high-level data, such +as interest rate, credit line, or owner. #+begin_src sql CREATE TABLE "Accounts" ( @@ -85,10 +74,10 @@ CREATE TABLE "Accounts" ( *Statements* -The Statements table uses the same unique identifier as the Accounts -table, meaning you can join the tables to find a monthly statement for -any of the accounts listed in the Accounts table. Each statement has an -account ID, statement date, and total balance. +The Statements table uses the same unique identifier as the Accounts table, +meaning you can join the tables to find a monthly statement for any of the +accounts listed in the Accounts table. Each statement has an account identified +(ID), statement date, and total balance. #+begin_src sql CREATE TABLE "Statements" ( @@ -103,10 +92,10 @@ CREATE TABLE "Statements" ( *Payroll* -The Payroll table is a separate entity, unrelated to the Accounts or -Statements tables. This table contains all information you would find on -a pay statement from an employer. As you change employers or obtain new -perks/benefits, just add new columns to adapt to the new data. +The Payroll table is a separate entity, unrelated to the Accounts or Statements +tables. This table contains all information you would find on a pay statement +from an employer. As you change employers or obtain new perks/benefits, just add +new columns to adapt to the new data. #+begin_src sql CREATE TABLE "Payroll" ( @@ -141,23 +130,18 @@ CREATE TABLE "Payroll" ( #+end_src ** Python Reporting -:PROPERTIES: -:CUSTOM_ID: python-reporting -:END: -Once I created the database tables and imported all my data, the only -step left was to create a process to report and visualize on various -aspects of the data. -In order to explore and create the reports I'm interested in, I utilized -a two-part process involving Jupyter Notebooks and Python scripts. +Once I created the database tables and imported all my data, the only step left +was to create a process to report and visualize on various aspects of the data. + +In order to explore and create the reports I'm interested in, I utilized a +two-part process involving Jupyter Notebooks and Python scripts. *** Step 1: Jupyter Notebooks -:PROPERTIES: -:CUSTOM_ID: step-1-jupyter-notebooks -:END: + When I need to explore data, try different things, and re-run my code -cell-by-cell, I use Jupyter Notebooks. For example, I explored the -=Accounts= table until I found the following useful information: +cell-by-cell, I use Jupyter Notebooks. For example, I explored the =Accounts= +table until I found the following useful information: #+begin_src python import sqlite3 @@ -182,12 +166,9 @@ df.groupby(['AccountType']).sum().plot.pie(title='Credit Line by Account Type', #+end_src *** Step 2: Python Scripts -:PROPERTIES: -:CUSTOM_ID: step-2-python-scripts -:END: -Once I explored enough through the notebooks and had a list of reports I -wanted, I moved on to create a Python project with the following -structure: + +Once I explored enough through the notebooks and had a list of reports I wanted, +I moved on to create a Python project with the following structure: #+begin_src txt finance/ @@ -212,16 +193,15 @@ This structure allows me to: 1. Compile all required python packages into =requirements.txt= for easy installation if I move to a new machine. -2. Activate a virtual environment in =venv/= so I don't need to maintain - a system-wide Python environment just for this project. -3. Keep my =notebooks/= folder to continuously explore the data as I see - fit. +2. Activate a virtual environment in =venv/= so I don't need to maintain a + system-wide Python environment just for this project. +3. Keep my =notebooks/= folder to continuously explore the data as I see fit. 4. Maintain a local copy of the database in =src/= for easy access. 5. Export reports, images, HTML files, etc. to =public/=. -Now, onto the differences between the code in a Jupyter Notebook and the -actual Python files. To create the report in the Notebook snippet above, -I created the following function inside =process.py=: +Now, onto the differences between the code in a Jupyter Notebook and the actual +Python files. To create the report in the Notebook snippet above, I created the +following function inside =process.py=: #+begin_src python # Create summary pie chart @@ -258,11 +238,10 @@ Other charts generated by this project include: - Charts of account balances over time. - Line chart of effective tax rate (taxes divided by taxable income). -- Salary projections and error limits using past income and inflation - rates. +- Salary projections and error limits using past income and inflation rates. - Multi-line chart of gross income, taxable income, and net income. -The best thing about this project? I can improve it at any given time, -shaping it into whatever helps me the most for that time. I imagine that -I will be introducing an asset tracking table soon to track the -depreciating value of cars, houses, etc. Who knows what's next? +The best thing about this project? I can improve it at any given time, shaping +it into whatever helps me the most for that time. I imagine that I will be +introducing an asset tracking table soon to track the depreciating value of +cars, houses, etc. Who knows what's next? |
