During the epidemic, all of the teachers at the university had to switch to an online platform for the lectures. Some of them went with Zoom (as it was the trend), others went with Discord (my favourite), while the dark side of them went with Microsoft teams. Why is it dark? -Because it's official for a 9am lecture that I have to wake up especially for and every time I need to type my username and password in it, then why can't you just make the app remember them?┐(´ー｀)┌ -I have to sign in with the email of the uni and every time MS teams redirects me to the portal that doesn't store them. Imagine waking up 5 min before lecture with 20% of eyesight only working and then trying to type your login credentials. That's when I decided to make another visit to Python with an automation library called Pyautogui.
#Why?
I know it might sound weird, but I'm lazy ( ◡‿◡ *) when it comes to typing my username and password every time I have to attend an online lecture. It is like a secret vault, I need to open MS Teams, then choose my uni's account, then it redirects me to the portal where I have to type my full email address, then a password that is 17 characters long between special cases and numbers, and all of this is a lot for someone who just woke up. So as an extension to my Python automation projects (The twitter bot) I decided to make a new approach this time. Controlling my mouse and keyboard to specific images on screen to trigger each function.
#Steps
Getting my way around the library was easy because the documentation (ref) explains the main moving functions for mouse and keyboard. For mouse, you can move to specific x, coordinates, or find it by image of that object, then move the mouse to the center of it.
- Find MS teams on my desktop and start it
- Click on my account
- Find username and password section
- Type info in each one
- Click on sign in.
#How it went?
Taking the screenshot was easy (thank god for win+shift+s) for the parts that needed it, the icon of MS teams with the closest it can be so it won't get distracted by things around it and fail to find the image,the confidence can be lowered for easier recognition, same will go for rest of parts. The whole application is driven by the "FindImage()" function that takes the location of the image as a parameter to search for it on the current screen. If the image doesn't exist, then print to the console, until it is found, move the cursor to it for either a double click or a single click (depends on the found image). Something to keep in mind, once the application starts, cursor can move to random places, so there had to be a FailSafeException to stop it, going with the mouse to any of screen corners will trigger it, I gave it a time period of 1 sec between each move while searching for the next image. For example, if the portal takes a long time to load, the console would still show me a message that it is searching for it, not just stop.