aboutsummaryrefslogtreecommitdiff
path: root/content/blog/2022-03-03-financial-database.org
diff options
context:
space:
mode:
authorChristian Cleberg <[email protected]>2025-11-11 22:49:13 -0600
committerChristian Cleberg <[email protected]>2025-11-11 22:49:13 -0600
commit51a7a02f0c96d49b68fbcc155414c218207fa270 (patch)
tree845af8aad0e8769352efc02fcd1044eed9cc1ec1 /content/blog/2022-03-03-financial-database.org
parent7d3e80ebf1dc770eac0e21890b74f18ba2d15a6b (diff)
downloadcleberg.net-51a7a02f0c96d49b68fbcc155414c218207fa270.tar.gz
cleberg.net-51a7a02f0c96d49b68fbcc155414c218207fa270.tar.bz2
cleberg.net-51a7a02f0c96d49b68fbcc155414c218207fa270.zip
fix grammar in 2022 posts
Diffstat (limited to 'content/blog/2022-03-03-financial-database.org')
-rw-r--r--content/blog/2022-03-03-financial-database.org159
1 files changed, 69 insertions, 90 deletions
diff --git a/content/blog/2022-03-03-financial-database.org b/content/blog/2022-03-03-financial-database.org
index 40612b5..b82fccf 100644
--- a/content/blog/2022-03-03-financial-database.org
+++ b/content/blog/2022-03-03-financial-database.org
@@ -4,60 +4,49 @@
#+slug: financial-database
* Personal Financial Tracking
-:PROPERTIES:
-:CUSTOM_ID: personal-financial-tracking
-:END:
-For the last 6-ish years, I've tracked my finances in a spreadsheet.
-This is common practice in the business world, but any good dev will
-cringe at the thought of storing long-term data in a spreadsheet. A
-spreadsheet is not for long-term storage or as a source of data to pull
-data/reports.
-
-As I wanted to expand the functionality of my financial data (e.g.,
-adding more reports), I decided to migrate the data into a database. To
-run reports, I would query the database and use a language like Python
-or Javascript to process the data, perform calculations, and visualize
-the data.
+
+For the last 6-ish years, I've tracked my finances in a spreadsheet. This is
+common practice in the business world, but any good dev will cringe at the
+thought of storing long-term data in a spreadsheet. A spreadsheet is not for
+long-term storage or as a source of data to pull data/reports.
+
+As I wanted to expand the functionality of my financial data (e.g., adding more
+reports), I decided to migrate the data into a database. To run reports, I would
+query the database and use a language like Python or Javascript to process the
+data, perform calculations, and visualize the data.
* SQLite
-:PROPERTIES:
-:CUSTOM_ID: sqlite
-:END:
-When choosing the type of database I wanted to use for this project, I
-was split between three options:
-
-1. MySQL: The database I have the most experience with and have used for
- years.
+
+When choosing the type of database I wanted to use for this project, I was split
+between three options:
+
+1. MySQL: The database I have the most experience with and have used for years.
2. PostgreSQL: A database I'm new to, but want to learn.
-3. SQLite: A database that I've used for a couple projects and have
- moderate experience.
-
-I ended up choosing SQLite since it can be maintained within a single
-=.sqlite= file, which allows me more flexibility for storage and backup.
-I keep this file in my cloud storage and pull it up whenever needed.
-
-** GUI Editing
-:PROPERTIES:
-:CUSTOM_ID: gui-editing
-:END:
-Since I didn't want to try and import 1000--1500 records into my new
-database via the command line, I opted to use
-[[https://sqlitebrowser.org/][DB Browser for SQLite (DB4S)]] as a GUI
-tool. This application is excellent, and I don't see myself going back
-to the CLI when working in this database.
+3. SQLite: A database that I've used for a couple projects and have moderate
+ experience.
+
+I ended up choosing SQLite since it can be maintained within a single =.sqlite=
+file, which allows me more flexibility for storage and backup. I keep this file
+in my cloud storage and pull it up whenever needed.
+
+** Visual Editing
+
+Since I didn't want to try and import 1000--1500 records into my new database
+via the command line, I opted to use [[https://sqlitebrowser.org/][DB Browser for SQLite (DB4S)]] as a GUI
+(graphical user interface) tool. This application is excellent, and I don't see
+myself going back to the CLI (command line interface) when working in this
+database.
DB4S allows you to copy a range of cells from a spreadsheet and paste it
-straight into the SQL table. I used this process for all 36 accounts,
-1290 account statements, and 126 pay statements. Overall, I'm guessing
-this took anywhere between 4--8 hours. In comparison, it probably took
-me 2-3 days to initially create the spreadsheet.
+straight into the SQL table. I used this process for all 36 accounts, 1290
+account statements, and 126 pay statements. Overall, I'm guessing this took
+anywhere between 4--8 hours. In comparison, it probably took me 2-3 days to
+initially create the spreadsheet.
** Schema
-:PROPERTIES:
-:CUSTOM_ID: schema
-:END:
-The schema for this database is actually extremely simple and involves
-only three tables (for now):
+
+The schema for this database is actually extremely simple and involves only
+three tables (for now):
1. Accounts
2. Statements
@@ -65,9 +54,9 @@ only three tables (for now):
*Accounts*
-The Accounts table contains summary information about an account, such
-as a car loan or a credit card. By viewing this table, you can find
-high-level data, such as interest rate, credit line, or owner.
+The Accounts table contains summary information about an account, such as a car
+loan or a credit card. By viewing this table, you can find high-level data, such
+as interest rate, credit line, or owner.
#+begin_src sql
CREATE TABLE "Accounts" (
@@ -85,10 +74,10 @@ CREATE TABLE "Accounts" (
*Statements*
-The Statements table uses the same unique identifier as the Accounts
-table, meaning you can join the tables to find a monthly statement for
-any of the accounts listed in the Accounts table. Each statement has an
-account ID, statement date, and total balance.
+The Statements table uses the same unique identifier as the Accounts table,
+meaning you can join the tables to find a monthly statement for any of the
+accounts listed in the Accounts table. Each statement has an account identified
+(ID), statement date, and total balance.
#+begin_src sql
CREATE TABLE "Statements" (
@@ -103,10 +92,10 @@ CREATE TABLE "Statements" (
*Payroll*
-The Payroll table is a separate entity, unrelated to the Accounts or
-Statements tables. This table contains all information you would find on
-a pay statement from an employer. As you change employers or obtain new
-perks/benefits, just add new columns to adapt to the new data.
+The Payroll table is a separate entity, unrelated to the Accounts or Statements
+tables. This table contains all information you would find on a pay statement
+from an employer. As you change employers or obtain new perks/benefits, just add
+new columns to adapt to the new data.
#+begin_src sql
CREATE TABLE "Payroll" (
@@ -141,23 +130,18 @@ CREATE TABLE "Payroll" (
#+end_src
** Python Reporting
-:PROPERTIES:
-:CUSTOM_ID: python-reporting
-:END:
-Once I created the database tables and imported all my data, the only
-step left was to create a process to report and visualize on various
-aspects of the data.
-In order to explore and create the reports I'm interested in, I utilized
-a two-part process involving Jupyter Notebooks and Python scripts.
+Once I created the database tables and imported all my data, the only step left
+was to create a process to report and visualize on various aspects of the data.
+
+In order to explore and create the reports I'm interested in, I utilized a
+two-part process involving Jupyter Notebooks and Python scripts.
*** Step 1: Jupyter Notebooks
-:PROPERTIES:
-:CUSTOM_ID: step-1-jupyter-notebooks
-:END:
+
When I need to explore data, try different things, and re-run my code
-cell-by-cell, I use Jupyter Notebooks. For example, I explored the
-=Accounts= table until I found the following useful information:
+cell-by-cell, I use Jupyter Notebooks. For example, I explored the =Accounts=
+table until I found the following useful information:
#+begin_src python
import sqlite3
@@ -182,12 +166,9 @@ df.groupby(['AccountType']).sum().plot.pie(title='Credit Line by Account Type',
#+end_src
*** Step 2: Python Scripts
-:PROPERTIES:
-:CUSTOM_ID: step-2-python-scripts
-:END:
-Once I explored enough through the notebooks and had a list of reports I
-wanted, I moved on to create a Python project with the following
-structure:
+
+Once I explored enough through the notebooks and had a list of reports I wanted,
+I moved on to create a Python project with the following structure:
#+begin_src txt
finance/
@@ -212,16 +193,15 @@ This structure allows me to:
1. Compile all required python packages into =requirements.txt= for easy
installation if I move to a new machine.
-2. Activate a virtual environment in =venv/= so I don't need to maintain
- a system-wide Python environment just for this project.
-3. Keep my =notebooks/= folder to continuously explore the data as I see
- fit.
+2. Activate a virtual environment in =venv/= so I don't need to maintain a
+ system-wide Python environment just for this project.
+3. Keep my =notebooks/= folder to continuously explore the data as I see fit.
4. Maintain a local copy of the database in =src/= for easy access.
5. Export reports, images, HTML files, etc. to =public/=.
-Now, onto the differences between the code in a Jupyter Notebook and the
-actual Python files. To create the report in the Notebook snippet above,
-I created the following function inside =process.py=:
+Now, onto the differences between the code in a Jupyter Notebook and the actual
+Python files. To create the report in the Notebook snippet above, I created the
+following function inside =process.py=:
#+begin_src python
# Create summary pie chart
@@ -258,11 +238,10 @@ Other charts generated by this project include:
- Charts of account balances over time.
- Line chart of effective tax rate (taxes divided by taxable income).
-- Salary projections and error limits using past income and inflation
- rates.
+- Salary projections and error limits using past income and inflation rates.
- Multi-line chart of gross income, taxable income, and net income.
-The best thing about this project? I can improve it at any given time,
-shaping it into whatever helps me the most for that time. I imagine that
-I will be introducing an asset tracking table soon to track the
-depreciating value of cars, houses, etc. Who knows what's next?
+The best thing about this project? I can improve it at any given time, shaping
+it into whatever helps me the most for that time. I imagine that I will be
+introducing an asset tracking table soon to track the depreciating value of
+cars, houses, etc. Who knows what's next?