Hacker News new | past | comments | ask | show | jobs | submit login
Show HN: I saw this mind-blowing experiment, so I made a simple version of it (github.com/momciloo)
403 points by momciloo on Nov 25, 2023 | hide | past | favorite | 129 comments
Two browser windows (acting as socket clients) communicate their:

- Screen dimensions - (screen.width, screen.height)

- Window dimensions - (window.innerWidth, window.innerHeight)

- Window X/Y position - (window.screenX, window.screenY)

...or whichever calculation works best for you.

The original work by Bjorn Staal https://twitter.com/_nonfigurativ_/status/172732259457002734 used localStorage, but I found sockets more fun, because if tweaked a bit, this can be shared with friends.

Here's a demo of how it works and the codebase: https://github.com/Momciloo/fun-with-sockets/




That's a great demo! I'd be curious to see how this would work with multiple monitors.

Also, thank you for voluntarily acknowledging that you were directly inspired by somebody else and giving them credit - the software industry could use more people like you.


What, thieves?

Make your stuff from zero, like the rest of us!


Pretty sure this is tongue-in-cheek, given the commenter (pretty high profile dev who creates a lot of cool stuff and does not shy away from using other open source code!) :)


that was super cool of you. Before letting people make the incorrect assumption, you hit em off at the pass. good people right there


Aw I thought it would be obvious :( Thanks for clarifying, of course nobody makes anything without being inspired from something that already existed, at least to a degree.

I guess it's hard to accurately convey sarcasm online, thanks again!


No worries, it was pretty obvious to me


Standing on the shoulders of giants


It was immediately obvious.


https://twitter.com/_nonfigurativ_/status/172798055677504755...

Here the original creator is demoing the behavior on multiple monitors


This, or something similar, would be nice for managing layers in a paint program such as Krita, Inkscape, or Gimp. Could be implemented simply as tabbed panes within the encompassing application window, and whatever tab is selected is the layer that is active for the editing actions.


I'm old enough to remember a bunch of demos using window position/dimensions from back in the day. I remember one with a physics simulation. I can't remember if it was liquid or just a bunch of solids, but you could drop things from one window to the other.

IIRC you don't even need the sockets. One can simply use message channels between the windows. If a window opens child window, iirc it has special access over it (I say special, because usually different tabs/windows are isolated from one another) which might make a local-only version easy to implement too.


Browser Ball maybe? https://experiments.withgoogle.com/browser-ball


Ah yeah! Browser ball is a classic!


If you like this, you might also enjoy WindowKill, a video game similar to Asteroids that cleverly uses windows that can overlap and interact with each other.

You need to shoot at the window boundaries, too, or the window shrinks. Additional windows appear later in the game with boss enemies.

Gameplay video: https://youtu.be/7iP68FZWVxM


> The original work by Bjorn Staal https://twitter.com/_nonfigurativ_/status/172732259457002734 used localStorage, but I found sockets more fun, because if tweaked a bit, this can be shared with friends.

Anyone have a link to what this was? The tweet is gone.


It's missing a digit at the end, original tweet: https://twitter.com/_nonfigurativ_/status/172732259457002734... / https://nitter.net/_nonfigurativ_/status/1727322594570027343


Thanks


He gives the link on GitHub but here it is (video) https://www.linkedin.com/feed/update/urn:li:activity:7133171...


Thanks


Reminds me of a cool demo of this for playing pong with browser windows http://stewd.io/pong/


Could someone explain what this means ? ( I’m not even sure I understand the gif on the GitHub page ) I just see windows seemingly sharing data.


The core of the trick isn't the window-window communication as much as using browser APIs to expose the window coordinates.


I think it's some kind of multiple client, single server setup. The windows are the individual clients, which pass screen geometry info to the server. The server then serves content to clients with different layout that makes content appear as a single object across multiple windows.


Cool! Feels like whichever window has focus, that window’s rectangle should draw on top.


Isn't that just to illustrate that it isn't just transparency?


But… why does it lag? Shouldn’t this be trivial enough to be instant?


Well, it's in a browser. Between trivial mouse move and trivial object coordinate change, there are numerous system and application layers to relay event to your interpreted (or compiled) code, and then to deliver state change to screen.

Even if we move to native world, making two different windows do the same instantly is not always trivial. For example, there have been articles about GUI toolkits making it impossible to have smooth window resizing. If we make everything ourselves (say, immediate drawing in response to WM_PAINT), we can only hope that the system notifies windows quickly, and that performance available is enough to pump bitmaps to screen. Or, better, we can try to use system-wide software or hardware compositor, add layers for individual objects, and only change their coordinates. Then we rely on compositor being good enough to handle 60/120/144/whatever updates per second when it's needed.


Querying window position is laggy unfortunately. It's a browser feature.


I wonder if it would be more instant if they used postMessage api instead of going over network.


Any local network usage should be nearly instantaneous.

My personal theory about the lag is that positioning the window is not synchronized to the JS thread, so the window moves before the JS thread is aware, which means you see the square move after the window has already moved.


Ah, my bad. I didn't read the code and just assumed there was a far away server.


I assume you mean BroadcastChannel?

https://developer.mozilla.org/en-US/docs/Web/API/BroadcastCh...


Going over local network is more or less instant


Background windows/tabs getting a lower priority maybe a factor.


Probably because of websocket network lag and also, in my anecdotal experience, window position updates are sometimes strangely jittery


Why is it over a websocket? Can't it just share local data?


It says so right in the OP

> The original work by Bjorn Staal used localStorage, but I found sockets more fun, because if tweaked a bit, this can be shared with friends.


I might be wrong, but I think the point is to simulate a server-client architecture (to show that it works even for an actual web app).


those were my thoughts too

You can use Broadcast Channels as well as Service Workers, even WebRTC would use a loopback interface!


This ought to work across different browsers though, for what it's worth. But ultimately it's just a fun experiment.


That's a fun use of LocalStorage.

I used the same LocalStorage sharing technique to update the target window when the setting had been changed on a separate browser window. Just listen for the storage.onChanged event for the update.


I don’t think this is using local storage. There’s a web sockets based server that is sending clients the location of the other boxes as they are moving


You can use a BroadcastChannel to send messages between the Windows directly. This should reduce latency when working locally.


Yes exactly, using web socket is kind of over-engineering. See: https://developer.mozilla.org/en-US/docs/Web/API/Broadcast_C...


I love stollen. It's one of the few aspects of Christmas I actually look forward to.


This reminds me of a project I did in school, a multi-user web browser. You could both control the same window's size, scroll position, and other state on each other's screens. The intended usage was having 2+ windows side by side.


None of these screen properties should be available to websites.


Width and height definitely need to be. I can't imagine why absolute position were ever needed.


Viewport width/height: yes since thats unavoidable, window width/height: no, screen width/height: hell no.


The available API's on the web are very questionable. Feross has a web security lecture series on youtube where he showed off the following abomination. Enter at own risk.

https://theannoyingsite.com/


warning, this is really annoying. imagine you're infected with a ton of adware in the early 2000s you will be logged out of many accounts

here's someone testing it out on Windows a few years ago, but I assure you even in the latest chrome on mac, it's at least as bad https://youtu.be/estvbRSj4LU?si=gq6-l4pWLC9yXqap&t=109


Why would you keep minimizing things instead of closing them? The video just looks like one of those meant to frustrate the viewer into wanting to do it themselves. I blame the creator of that video as much as the site itself. The site itself served a purpose. The person running that video has a much less clear motive


I think if you close a window, another pops up. Or two.

You should go to this site and test this.


It spawned an app outside of the browser. That's the window being minimized that I meant. I'm not an unawares person to the site's purpose. You're being a bit disingenuous with your reading of my comment


I tried it on Safari private window. It was absolutely horrible. I hesitate to guess what would have happened if I didn't keep clicking cancel whenever it asked if I wanted to allow downloads, etc.


uBlock Origin has prevented the following page from loading:

    https://theannoyingsite.com/
Because of the following filter:

    ||theannoyingsite.com^$document
Found in:

    uBlock filters – Badware risks


It’s not malware, I promise. Just super annoying :)

Source code: https://github.com/feross/TheAnnoyingSite.


It lived up to its name for sure!!


holy shit, I was not expecting that LOL


This is cool. It reminds me of a research team I worked in a few years back that had built something like this for mobile devices. It used a camera overhead to provide info about device positions, allowing all the devices to act as a single screen on a surface [0].

[0] http://www.huddlelamp.org/#tech-video


what fresh pop-up/pop-under hell even is this?!? I specifically asked for not that


More like this:

- https://twitter.com/steveruizok/status/1727625036159234555

- https://twitter.com/_nonfigurativ_/status/172732259457002734...

- https://twitter.com/wesbos/status/1727755227766350031

- https://twitter.com/wesbos/status/1727722358465523721


I had some fun doing some SQL joins with this:

https://twitter.com/burcs/status/1728081692273823863


Okay this one is actually cool. I'm hoping it's not totally one-off scripted but it's fun either way. :)


Video of last one: https://youtube.com/watch?v=3Hye_47c0Pc


Here's something similar I made yesterday. Just view source of the controlling page and the pop-up pages for how it works (single page with inline vanilla JavaScript and no npm install nonsense). Just uses postMessage(). https://cms-compsci.deno.dev/mrgordon/window/


Does this work on Wayland?


Tested on Sway with both Firefox and Chromium. Resizing works (no surprise) but the position doesn't work. window.screenX/Y always return 0.


Good. Websites have no business knowing anything of the world outside the viewport.


The devs behind tldraw also played with this to create something cool - https://twitter.com/steveruizok/status/1727625036159234555


This and the original reminded me of this really old Chrome Experiments demo, and some others like it: https://experiments.withgoogle.com/browser-ball


Very cool. What's the working title for this interface / experience?


LocalStorage API.


Or a postMessage could be used if one tab is opened from the other.


Reminds me of http://www.thewildernessdowntown.com/ [desktop]


I’m kind of amazed the original version is using localstorage. Storage and retrieval are blocking operations. I’d expect render performance to be terrible.


correct URLs:

https://twitter.com/_nonfigurativ_/status/172732259457002734...

http://farside.link/twitter.com/_nonfigurativ_/status/172732...


made something similar 14 years ago...

https://youtu.be/bEwsuWNxdno

domain isn't mine anymore...


Oh wow, from December 2008.

Here's to no one trying to patent it now!


What API allowed that back then?


It was an API I developed using ActiveX on Microsoft's Internet explorer and NPAPI on other browsers


I don't see what is technically new about this? What API has been missing to do this?

Tcp and window.screenTop and screenLeft should get you there right? Hmm, yeah, 2008 is a long time ago though..


Two more examples of multiple windows as a single unit. This time on the video game side.

(Ludum Dare 35 Overall Winner) WindowFrame: https://youtu.be/CY7NUHn6GQg?t=273

Window Kill: https://youtu.be/7iP68FZWVxM?t=160


Metal Marines, a 1993 game with multiple windows that interact with each other: https://youtu.be/KiiAUP1fG14?t=297


Thanks! I am adding this one to my list!


The mind-blowing part is not that this works, but that it is secure.


Love how simple the code is, with almost no external dependencies


I'm wondering about practical application of this demo...


I have always been fascinated by those sci-fi movies in which a person would swipe up on their tablet, and the current window would fly to another monitor.

This maybe be something that can achieve that.


It would not surprise me if the original practical application is long gone, but it's now used to fingerprint browsers for ad serving purposes.


Maybe making colour mixing ven diagrams?


I love the concept and the demo! Amazing creative work


Really great. The web is really going forward


Seems like this was all possible 10+ years ago with existing APIs at that time. I don’t see what’s so special tbh.


W-why is window LOCATION exposed to websites? Apparently even in FF private browsing...

At first I thought this is an electron only thing. I get it that we gave up on fingerprinting resistance but this is like pissing on it's grave.

EDIT: privacy.resistFingerprinting set to true fixes the coords to 0


I thought you meant literal `window.location` and was confused, but yeah `window.screenX/Y` is a pretty suspect feature, now that you mention it.


I'm as surprised as well.

I'm trying to guess why, and can only imagine that at some point the idea was to support multi-window webapps, like maybe you'd have separate windows for documents and different tool palettes, and they'd want to know each other's locations to try to prevent overlaps? I mean, back in the day, framesets were another idea that experienced some popularity before being totally eclipsed by <iframes> -- all of this back in the era when everything was based on the multiple-windows paradigm rather than the multiple-tabs paradigm. Window.screenLeft apparently originated in IE [1].

Just a guess though. Very curious if there's any legitimate reason for it to continue to exist, or whether W3 ought to deprecate it.

[1] https://developer.mozilla.org/en-US/docs/Web/API/Window/scre...


On Wayland, a native window can't even get that information. At least not in a way that works across composiotors.


Am I the only one that's horrified that window.screenX and window.screenY exist? This is not information that I want to provide.


Yep. There are some ridiculous web APIs out there that expose all sorts of things you'd think should remain private. Take the Battery Status API for example [1].

[1] https://developer.mozilla.org/en-US/docs/Web/API/Battery_Sta...


IIRC, that one and a number of other lower-level “WTF” APIs were introduced for Firefox OS. They made a little more sense in the context of providing an interface for mobile phones using the browser stack.


This would be useful in WebXR (given opt-in)


Using the demo in https://developer.mozilla.org/en-US/docs/Web/API/MouseEvent/... it seems Firefox pretends the window is always at (0,0). So that's nice.


You likely have `privacy.resistFingerprinting` set to true.


as everyone should!

people forget lesson from recent past. i don't know when google strong armed w3c to include this back... but we had this since ie4 days and then disabled everywhere since it was abused left and right for click jacking attacks.


> as everyone should!

Not until Mozilla stops telling websites that I want a light color scheme if I turn it on. privacy.resistFingerprinting does too many things and you shouldn't need to use that sledgehammer to prevent the browser from leaking information that no website should know.


Sorry, i'm a little lost. I tried thinking about it, but I can't think of any nefarious attacks someone could do with screenX/Y. Why is that such a risk? What could they possibly learn from that other than screen size?


The fingerprinting potential may well be limited, but it's the principle of the thing: The browser should not provide unnecessary information about its surroundings.

It would be another matter if this was a very useful feature, but I don't think that it is.

Here's one use for it: Ads could sense that they are off-screen and withhold content until the ad is moved back in view. I could easily see screenX/Y being used for that. I mean, I hope not, but how am I going to stop it?


knowing where the window is, you can generate something the user will click on a known position and then at the right time you trigger something like an administrator escalation privilege confirmation dialog and the user clicks that instead.

click jacking is always fun.


Because fingerprinting. Shit like this is the reason you are supposed to use the Tor browser only with a tiny window.


> you are supposed to use the Tor browser only with a tiny window

I hadn't ever heard of this. Here is a statement from the Tor project for anyone else wondering what this is about:

> We automatically resize new browser windows to a 200x100 pixel multiple based on desktop resolution which is provided by a Firefox patch. To minimize the effect of the long tail of large monitor sizes, we also cap the window size at 1000 pixels in each direction. In addition to that we set privacy.resistFingerprinting to true to use the client content window size for window.screen, and to report a window.devicePixelRatio of 1.0. Similarly, we use that preference to return content window relative points for DOM events. We also force popups to open in new tabs to avoid full-screen popups inferring information about the browser resolution. In addition, we prevent auto-maximizing on browser start, and inform users that maximized windows are detrimental to privacy in this mode


Fingerprinting by screen size is real. I've checked one day, and it turned out my browser viewport size when maximized is super unique (like <0.1% users unique). That's mostly because I use sidebery (tabs on the left instead of the top) and don't have a (visible by default) bookmarks tab, but wow I didn't expect to be so obvious for advertisers by just reading my page-usable screen size.


I just checked, it is possible to override them, which causes them to stop updating when you change the your window's location on the screen.


Use BroadcastChannel; you don't need no sockets

https://developer.mozilla.org/en-US/docs/Web/API/Broadcast_C...


Nice, my first thought was to use LocalStorage. Didn't know about BroadcastChannel.

Sounds like OP wanted to use sockets for fun though.

>The original work by Bjorn Staal used localStorage, but I found sockets more fun, because if tweaked a bit, this can be shared with friends.

Who among us hasn't overcomplicated things for fun? :)


Might as well link to the original / "stolen from" author's equivalent demo with Three.js (open this in multiple overlapping windows) - https://bgstaal.github.io/multipleWindow3dScene/ - and repo - https://github.com/bgstaal/multipleWindow3dScene/


3 days old, twitter one seems 4 days old, who stole from who?

Anyway nice repo, thanks for sharing!


The links I posted are from the original Twitter guy but a simplified version he made to open source it


The original seems to be on LinkedIn. Does anyone else find it weird when geek content is posted to LinkedIn? It's the last place I'd think to look. Am I missing out or is this an outlier?


I work in computer vision and like hearing about the latest thing. I have been finding LinkedIn increasingly useful as a way to learn about machine learning.

There are _a lot_ of posts that I have to scroll through of little to no value, but every so often I see a post by Yann LeCun or the lead developer advocate at TensorFlow or a paper someone has liked that catches my eye or a new model release.

"Machine learning LinkedIn", as it were, is a real thing; not sure whether this applies to other disciplines.


Yann posts mostly on X.


I've recently started posting my content on LinkedIn (usually just linking to my blog or publication with a short explanation). Main reason is that I'm a bit unhappy with the new Twitter (a place where I used to share the most), approximately nobody reads Mastodon so I can't stop there, and all least I already have LinkedIn so it's low cost to share there.


For sure. It’s approximately the first time I’ve seen something like that on LinkedIn.


Wow this seems profound, I didn’t even know you could do that, that could be the future of augmented reality if you think about it, layers! Great job!


Is it really different from any multiplayer game? Two view ports, shared coordinates, independent renderers. The “trick” is that the view ports are overlapping, but I would imagine sendMessage might be faster and not require any server round trip.


I was just trying to be nice, sheesh. happy holidays


Could you please expand on your "if you think about it"?


Spoiler: he didn't.


sure, if you think about the long term usage of augmented reality then we will need to have multiple layers of content {rationale=because we need to focus on one thing in the context of multitasking, because we want to reskin the environment while still maintaining the content, and to avoid unsafely obscuring the field of view during ambulation and manipulation tasks}




Consider applying for YC's Fall 2025 batch! Applications are open till Aug 4

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: