Hello!😄 This blog is going to be a little different as I won't just talk about the things I did this week, but also some of the things I've learnt during this experience.
The biggest and most significant change (or rather feature) I made to the add-ons was caching results. I decided that the YOLOv3 416 model is the best model to ship with the object detection add-on given its accuracy, size and latency. However, it was significantly slower than the tiny-YOLOv3 model. This meant that both the object detection and image captioning add-ons took a minimum of 5 seconds to produce results. It made no sense to wait for the same result again so I needed to cache results for the session. Initially, I tried to determine if the detection process was started on the same image by using the navigatorObject. However, this was a dead-end since they are not uniquely identifiable. So I tried creating a hash out of the image and that worked like a charm! here's the code for it:
rowHashes = []
for i in range(imgInfo.recogWidth):
row = []
for j in range(imgInfo.recogHeight):
row.append(pixels[j][i].rgbRed) # column-major order
rowHashes.append(hash(str(row)))
imageHash = hash(str(rowHashes))
Pretty simple! pixels is just a 2D array of RGBQUAD objects that store the image pixel data as [rgbRed, rbgGreen, rgbBlue, rgbAlpha]. The in-built hash() function is responsible for all the magic here but it can only work with immutable data types so I needed to convert my lists to strings. Just one colour channel was sufficient so I went with the red channel. I hoped that I wouldn't have to go through all the pixels to produce a hash but some images have padding and so just a portion of the image wouldn't suffice.
Now to talk about the things I've learnt... I think this message by Joseph on the NVDA developers group pretty much sums it up. I started this internship with the idea that this profession is all about the code. With zero prior experience in open-source development and having only worked on projects that I found "cool" or "interesting". But during this internship, I've come to realize that there are other, more important things about this profession. Writing code that (potentially) has real-world impact, working together with other developers and even having my code reviewed and my ideas and beliefs questioned/challenged have been a real eye-opening experience. Looking back, it doesn't seem like I've written much code at all. Just a few thousand lines over the past two and a half months. But what's worth measuring is what I've learnt and not the number of lines of code written.
To list a few things...
I've learnt how to write and submit pull requests. Starting from something as simple as fixing a typo to changes I wished for in the source code that would also benefit others out there.
I've learnt that the comments you write are not just written so you understand what your code does when you come back to it but also for any other developers who might work with your code. You don't just explain what's happening, but why you made some decisions over others. This prevents others from trying things you know don't work.
I've learnt to explain my choices and justify them to not just myself, but also to my mentors and the developer community at large. You pull requests aren't just about the changes/feature you've made, you also need to explain, justify and convince reviewers of all that you have learnt/tried while working on the pull request. If you can't do that, all your efforts might be for nothing.
I've learnt to weigh choices and prioritize them based on the value they produce. It might be fun to start working on a new feature but it is better to fix a bug that is giving users a bad experience. On the other hand, it might be tempting to fix a bug that rarely ever occurs but it is better to work on a feature that will be very useful to more users, more often.
I've learnt to put myself in the shoes of the user and think about what they would like while developing code instead of thinking about what I would like. This has been especially difficult as a sighted developer writing code to be used to by blind users but I feel like I'm better at it than I started.
I am thankful that my mentor Reef has been kind and patient with me, nudging me in the right direction and guiding me into thought processes that will yield the best results. I am glad he chooses to teach me how to be a better programmer and doesn't just tell me what I need to do.
Yes, I've learnt a lot but I have still got lots to learn. I don't just have to get better at all the things I've listed above, I also need to improve the way I code. Writing code that works is great, but being able to have the right processes that result in good code might be better. I also need to contribute by helping others and reviewing their work.
Until the next blog post👋