chrome extension to record and play window events

Summary

I am looking for a Chrome extension that will give me JavaScript commands called something like “ListenForWindowEvent()” and “PlayWindowEvent()”, which I can use to record raw window mouse and keyboard events, and replay them later. I don’t want an extension that provides its own UI for recording and replaying sequences of events. I just want to be able to access (detect and play) such events from my own JavaScript code.

Details

Using JavaScript (with or without JQuery), I can track events (mousemove, mouseup, mousedown, click, keyup, keydown) almost anywhere on a page, and I can “replay” these events, mostly, by triggering the appropriate events (e.g., “click”) on the appropriate elements.

However, my understanding is that there are certain events that I can’t quite detect or simulate, which all seem to have to do with “system level” components on a web page such as select (drop-down) menus, text input boxes, and scroll bars.

For instance, I don’t believe with just JavaScript that I can detect that the user dragged a scroll bar from position A to position B. I can listen for the scroll event after the fact, but can’t detect the lower-level mouse events that caused the scroll. And even if I had these events, I couldn’t replay them by causing the same click/drag events on the scroll bar. (I can only set the scroll value of the element.) Similarly, I can detect when a select menu changes value, but I don’t have access to the mouse events that popped the menu open and then selected a particular item. For text input boxes, I believe I can see the individual key up and key down events, but I can’t replay them into the box — again I can only set the value of the text input element as a whole.

Assuming all of that is more or less correct, what I want is a Chrome extension that would give me access to those low-level events (both monitoring them and replaying them) via JavaScript. I see that there are plenty of extensions that allow recording and replaying of these low-level events on the browser window. But these all seem to come with their own record/replay GUI. I don’t want to use someone else’s GUI. I just want to be able to record and replay these low-level window events myself via a simple extension that gives me JavaScript commands like “ListenForWindowEvent()” and “PlayWindowEvent()”.

I know that Puppeteer is Google’s way of playing low-level events to a window. But if possible I’d like to avoid having to use a heavy solution like that (and I don’t even know if Puppeteer can be used to record the events in the first place?). I imagine I can start with the source code of one of the standard record/replay automation extensions and make my own simpler extension. But I wanted to know if such an extension already exists. My Google searches only found the full-GUI ones.

What are the ways to do web automation with a Chrome extension?

What I want to do is, develop an extension that adds a button to a site, and when a user clicks on that button, the extension should start doing some web automation, i.e. go to another website and complete a certain task there. However, that whole process should not be visible to the user and should happen in the background. My question is, is it possible to do this with any of Chrome’s APIs (and if so, please point me to it), or do I have to use a web automation library to do this? E.g., since I am using JavaScript, I’d import this automation library https://github.com/segmentio/nightmare into the chrome extension, and do the automation with it. I’m just thinking there should be a cleaner way than this one.

Just to clarify, the idea here is not to do something in the user’s browser without their knowing about it. It’s the main and obvious use of the extension.

Content Settings in Chromium Browser

Is there a way to get/set the USB Hardware ID’s for Mic/Camera in Chromium browser from the command line.

I could not able to find out. The requirement is to set a particular Camera/Mic during runtime without manually updating in chrome://settings/content in the chromium browser.

It will be great if I can get some link or some sample code. Thanks !!!

How to pass pass user credentials to chrome via Native Messaging API

This is a pretty specific problem to have … but if you’re using Selenium, etc, from a machine which is not part of an Active Directory and you’re being foiled by browser popups, I have a solution for you.

I will explain the problem and link to some resources (and my other questions, with even more links in them) which informed the solution then I’ll post the change I made to the example for the extension I “wrote.”


Problem:

You’re automating or testing via selenium or something similar … and an auth popup comes out of the blue! But this popup isn’t JavaScript and you’re required not to save any credentials on the machine you’re testing from.

How do you pass through the authentication credentials to the browser and prevent that popup from occurring … but without using keystores, browser storage or, ghasp, a file?

Once you know how to pass that data in, how do you then get the values into the browser in such a way as to allow hands-free authentication?

Python: Pass arbitrary values to js function in chrome extension at runtime

I’ve been working on a fairly specific problem with automation, described in this question

Note that I’m using python with Selenium WebDriver

I have since uncovered a maze of changes and problems … suffice to say I’m attempting now to use Chromium and build an extension to deal with the authorisation prompt. The inspiration for this approach comes from here.

function callbackFn(details) {
  return {
      authCredentials: {
          username: "[insert_username_here]",
          password: "[insert_password_here]"
      }
  };
}

chrome.webRequest.onAuthRequired.addListener(
    callbackFn,
    {urls: [""]},
    ['blocking']
);

Note the proxy parts are completely useless to me – I am interested in the listener and the callback function.

My intention is to use the same process as Mike used but with one very important change: I want to pass a json (or anything, really) with the username and password directly to the callback function.
I am not particularly great with javascript so I accept that I could be missing something very obvious.

So how can one pass arbitrary values (in this case, 2 strings) to an extension at runtime using python and/or SeleniumWebdriver?

Thanks in advance!

how can we automate chrome extension that opens in chrome legacy window?

how can we automate chrome extension that opens in chrome legacy window?

how can we find elements on UI of chrome extension that opens in chrome legacy window?

How to access the values of Chrome’s Dev tools Network tab’s Request or summary using Selenium in python/java?

Im using chrome option to access the performance logging using selenium, Im trying to write a code that would help me figure out the total number of http request and the size of the page after after loading is finished. Manually we can check this by using Dev tool’s network tab . Just need to know how to access the Network table’s value or summary values .Because the performance logging is not giving me the summarized values that i need. I would like to write a code to get

Total number of requests=
What is total weight of the Page =

If it is possible .

Screen shot of network tab highlighted the Summary and Request table values that i need to access

capabilities = DesiredCapabilities.CHROME
capabilities['loggingPrefs'] = {'browser': 'DEBUG'}
capabilities['loggingPrefs'] = {'performance': 'ALL'}
capabilities['perfLoggingPrefs'] = {'enableTimeline': 'true'}


driverLocation = "/Users/harisrizwan/Selenium/chrome/chromedriver"
os.environ["chrome.driver"] = driverLocation
chrome_options = Options()
chrome_options.add_argument("headless")

driver= 
webdriver.Chrome(driverLocation,desired_capabilities=capabilities)
driver.implicitly_wait(10)
driver.maximize_window()
baseUrl="www.google.com"
driver.get(baseUrl)

using pandas to create a dataframe of the log.

df = pd.DataFrame(driver.get_log('performance'))
df.to_clipboard(index=False)

Thank you .