Captain’s log, stardate 8.23.2021.
It’s been over 7 weeks since my last post on this blog. Since then, a lot has changed. This is meant to be a short summary of what’s happened.
First and foremost, I made an important decision regarding the project. Up until about a week or two ago, I was still hoping to implement my own version of the network proposed by Sengupta, et. al. in the all-important paper that this madness is all based on. I realized my reasons for wanting to train my own version of it were no longer valid. When I began this project six months ago, I was still exceptionally brand-new to ML and PyTorch, and so implementing my own version of Sengupta’s architecture was primarily meant as a learning exercise. I dedicated a ton of time and effort to it, and learned a lot from doing it, but I now feel that I don’t have much more to gain from it. I was never able to get my version of their architecture to perform up-to-spec, anyways.
Thus, from now on, the matting part of the project will be handled using pre-trained weights made available under the MIT license by Sengupta and his team at the University of Washington. Personally, I am glad that I’ve been able to swallow my pride and acknowledge that the best way forward for this project is to not try to do everything myself. Instead, I need to be focusing on only the parts of the project that haven’t already been cracked, and it turns out there’s still a lot of problems that need solving.
So, what am I actually working on these days? Well, to put it briefly: lining up images automatically. Turns out it’s a really hard problem in this particular context, and I’ve already implemented and tested several architectures and solutions to try to get it working. Most recently, I’ve been diving into deep homography estimation. We’ll see if that actually bears any fruit. This coming week, I plan to look into recurrent homography estimation, and then if that doesn’t work, probably recurrent TPS transform estimation. We’ll see.
There’s a lot up in the air right now, but rest assured I am still consistently chipping away at this problem and having fun while doing it. I genuinely believe that I can get this system working, but it’s going to take some work.
Thanks for reading,
Kai.