Three years ago, when I first returned to Oracle, one the SEs I managed complained to me that we didn’t have a good way of sharing information as a team. While I agreed with him, I also didn’t have a good solution. And thus began an epic saga that took three years and resulted in a crazy, distributed Obsidian.
But first, how did we get there?
Just do this… uh…
The problem seemed easy. Just share create a shared knowledge base, let people have access, and they’ll contribute. Yay!
Well, not quite. We created a shared knowledge base in Confluence, gave the team access, and spent a few weeks contributing articles. We got 20 or 30 decent pages and then no one contributed any more. The confluence site required us to be on VPN, which most of the team wasn’t on a regular basis. It was also outside of our normal workflow and it didn’t have any of the documents we used regularly.
So it withered and died.
A few months later we had the grand idea to “just collect the good stuff” and proceeded on a quest to get all the best content from our team, put it in a shared repository, and let everyone have access. Awesome – a knowledge base!
Except that again, it was outside the normal workflow and mistakes were made easily. More importantly, the folder structures were messy. Was that document for a B2B company or B2C? What if they’re both? Do we categorize according to product, industry, presenter, time, or something else?
It ended up the only person who could find anything was the person who put the repository together. And thus, it stopped being used except as a place we pointed brand new people.
But, Oracle’s internal document sharing product (Oracle Content and the internal version is OraDocs) also comes with a web page builder that can access the files. Awesome! We’ll build a portal!
And thus, yet another epic quest and a few months of work by a few people. It resulted in a nicely curated site with links to past work done in different categories. Except that it was again outside of our day to day workflow, wasn’t regularly updated, and could only be updated by a few people. It died with the exception of being a nice place to send brand new people. No one used it and it got out of date in six months.
Now what? Notes?
Eventually I realized the core issue was that our knowledge base had to be part of our day to day work or it would never be used. If something isn’t part of the workflow and there isn’t a clear trigger for when to update it, nothing will happen.
I eventually settled on the idea that we needed to be sharing everything, not just our best stuff. We could always cull and curate, but it’s hard to point to great things if you don’t have the item in the first place.
So I started down the path of creating a shared note taking space. I explored OneNote (unable to use on Mac due to security risks Oracle didn’t like), TiddlyWiki, RoamResearch (not encrypted at the time, not Oracle approved), Logseq, and eventually Obsidian. The process took a long time and – though I personally use RoamResearch for most of my daily notes – we also had the constraint of a locally hosted, non-SaaS solution in order to get through Oracle’s security controls.
Interestingly, as I made my way into shared notes, I came to realize that most notes don’t need to be shared or don’t really need to be shared in the same format. They just need to exist in a place people can find them.
The Solution
The team’s final solution was a combination of Oracle Content (on OraDocs, the internal version) and Obsidian. With Obsidian being locally hosted, we could control most of the security vulnerabilities without relying on third party hosting of secure data. Additionally, the whole team was already using OraDocs to store many of our files.
Oracle Content
By focusing on creating shared folders for every one of our projects in a similar format (thank you Post Haste), we had a consistent structure that made it easy to find work in progress. It also became easy for the team to move their work off of their local machines and push it to Oracle’s cloud storage. With Oracle Content, they had both a local working copy and a copy in the cloud shared with the team. So the team started doing all of their work in the shared folders.
Immediately, the process simplified collaboration across the team. Anyone on the team could just share a link, the other person could jump in and get a read on the project so far, and then could contribute immediately. It sped up our ability to work together immensely.
Previously, we would share files through Slack or email, send different versions of the documents back and forth, and constantly try to figure out who had which version. It was a mess. Plus, when people left the company, most of their files left with them. It’s really hard to go back to see what was presented to a client last year when the person who did the presentation is no longer at the company.
Obsidian for Context
However, we still had two problems – the folders lacked context and we didn’t have a good way to share information that should be in a knowledge base.
That’s where Obsidian was brilliant. It creates markdown files that sit inside of a vault. Those files are indexed and easily searchable, interlinked, and referenced from other documents.
By setting up our vault correctly, we could get the context we needed out of the CRM, reference the shared folders, and create a knowledge base the team was working in on a daily basis.
Additionally, by using good tagging, we could easily writing Dataviews to surface different deals and related files across the vault.
Version One
The first version of our knowledge base put the Obsidian markdown files directly inside of the folders for each opportunity. That made linking to folders very easy and was simple to just Slack someone an obsidian link to get them both the background and the link to the folder the other documents (Excel, PPT, etc) were in.
The problem was that Oracle Content didn’t allow us to selectively sync folders. So everyone had the same .obsidian folder and were constantly overwriting each others preferences. It was untenable.
Instead, we used Syncthing to share the files and created a server that was always online as a relay in case people weren’t online to get the files (we work across timezones and people travel). It worked great, but wasn’t really within the standards of Oracle so we kept it small.
Version Two
Recently, Oracle updated Oracle Content to allow the selective syncing of folders. That let the team choose to sync our core folder, but unsync the .obsidian folder so everyone could have their own preferences. It worked great.
Well, it was great until everyone started really using it. Then suddenly we were blowing up hard drives. When the shared folder starts being 30-50GB, a lot of people ran out of space really fast. And then deleted TONS of files from the shared drive. Oops.
I was able to recover the files – Content has a good trash management system – but we had to solve the scale issue. And thus, pointer files.
Instead of having the markdown files for each opportunity in the shared folder, we put them in a separate file called ‘Index’. Our build process then queried the Oracle Content API to search for any folders with the Opportunity ID in the name and linked those folders as a list in the Index markdown file. That way the team could easily access the index without destroying their machine and selectively download the folders they were working on at the time.
Build Process?
Yes, I wrote a few thousand lines of python to make this whole contraption work. Fundamentally, it was just taking a few reports from our CRM, breaking them into markdown files, and ensuring they were properly tagged/linked together using obsidian links.
The underlying vault ended up with reference files for Deals, Accounts, People, Industries, Partners, Competitors, and Products. Each of the reference files used Dataview to create tables listing the deals a particular person, competitor, product, etc was involved in with a link the file.
The ultimate result was really fast. You could navigate across the markdown CRM, find what you needed, and then end up in the folder with the related assets in a few seconds. Deals were either “Active” (currently being worked) or “Past” (closed). Need background information on a deal? Go look at all the deals we’ve worked on that account in the past (Past -> [account folder] -> deal 1, deal 2, etc). Looking for a deal we’re actively working on? Go to Active -> [account name] -> [deal name] and you’ll find everything you need.
For a few weeks, the process was working great across about 40 people. It had worked the previous year with my team of 6 exceptionally well – allowing us to collaborate, have shared notes, and understand everyone’s weekly work more easily. The larger team was just starting to get the process going before our group was radically reduced in size. The team that is left continues to use it, though in a smaller scope now.
Caveats
There are a few issues with using Obsidian in a widely distributed manner.
Concurrency and overwriting
First, there is almost always lag time in the file updates. That creates clash issues. Either your sync mechanism is unaware of the clashes and simply overwrites constantly or you have an issue with diff merges. We started by trying to automate diff merges and then realize there was a better way.
In larger files where multiple people were likely to be editing concurrently, we broke up the index page into smaller pages that were instead referenced from the index. For example, in our weekly progress updates, instead of having everyone edit the same page, we used the “Weekly Update” to reference a page each person updated. By using the ! sign, we could hot load the file into the index page (ie, ![[Cody Boyte Update – 08/09/2022]]). That let each person create updates and the updates would show up as soon as the sync happened without breaking the content anyone else edited in the meantime.
Private information
The only way we could get the process to work was to make the entire folder structure public to our team. Otherwise Obsidian can’t properly index the files to ensure consistent naming without duplicates. While there are some solutions for semi-private data like encrypting some of the content with a key, that wouldn’t really work for our team.
Instead, we just kept secret things elsewhere (see: password managers, other note systems, etc). It’s a shared folder for the team, not a place for things that shouldn’t be seen by others.
Likely scale issues
I’m not sure that this process would be scalable much beyond 30-50 people. It certainly wouldn’t scale to hundreds of people using it concurrently. The number of files being synced could get crazy fast and duplicate naming issues would abound.
I’ll return to this post a little later to add some of the stats on the size of the vault (~5k markdown files, 40GB+ of shared files, etc), but let’s just say it was big with a lot of people using it at the same time.
Would I Do This Again?
Yes, in a heartbeat. It’s the first system I’ve used that made it easy to navigate our CRM for the solutions engineers. We didn’t need to add info, we just needed to see it. All of our files were at our fingertips and easily shared. Plus, we were able to quickly share weekly updates, reference deals, send links to the team with all the context they needed to help, etc. It was a lot of work to make it function correctly, but when it did the process was a bit magical.