Sweet, it's all built with React and Redux! And there's some _very_ hefty reducer logic in there.
I'd love to chat with some of the Uber engineers who worked on this. I'm a Redux maintainer, and I'm always eager to hear feedback from people who are building serious applications with Redux.
Reducers are almost always the first thing I dig into whenever I open up a Redux project. Tells you what the state structure is, how it's being updated, what sorts of actions are being used, etc.
Only looked through the code for a few minutes, but a couple things I noticed:
- You've got an interesting wrapper around `connect()`, which includes something called `withLocalSelector()`. Can you clarify what that does, and what the other use cases are for wrapping up `connect()` ? I think I also see some other assorted selector functions being used as well.
- The couple reducer files I looked at are pretty large (several hundred LOC), and appear to be working with state that's nested 3-5 levels deep. A good portion of that appears to be normalized, per our recommendations. Have you run into any particular issues managing state that's nested that way? Have you considered any approaches for simplifying some of that logic, such as "reducer composition" (delegating the work of updating more nested state to their own reducer functions), or using an immutable update library such as Immer?
Also, the blog post talks a lot about the dataset sizes that you're dealing with. Any particular performance issues you've run into, and if so, how have you dealt with those?
Finally, any particular pain points you've experienced? Any things we can do to improve the docs (like better teaching certain patterns), or issues you'd like to see officially recommended solutions for?
If you'd like to chat about this in more detail, feel free to ping me @acemarke over in the Reactiflux chat channels on Discord.
- withLocalSelector() allows us to mount multiple kepler.gl component instance in the reducer and select which one to dispatch and read from. We wrap around connect(), so the user doesn't have to worry about it. They can mount as many kepler.gl instances as they want and it should just work.
- I know nested state has performance drawbacks, that's why we try to flatten certain properties with large arrays, such as datasets, layers, and layer data. But for other properties that are simple strings, floats or boolean, I think it's fine to have them nested. Because we have this reducer plugin feature, where developers can add custom reducer handlers, I try to keep the number of reducer small, so it's easier to explain what they each do. Besides that, I came across couple of cases where 1 action updates multiple reducers sequentially, I don't want to add thunk, so we then had to add this composer function which is kinda complex.
There are lots of questions, don't have time to answer them all, but I would love to keep this conversation going. :)
Don't know if anyone else has tried this, but I can't seem to get it to load a file. It spins even on the smallest csv I can try, but in the background I can see that it has the file name on the GUI. something I'm doing wrong?
Possibility because your csv format is different. Does it have a header? etc. If you file an issue on github and link to your dataset https://github.com/uber/kepler.gl/issues. I will take a look. Thanks for trying kepler.gl!
I think it had something to do with starting the server in an SSH session. I restarted it through an X session, and forwarded the port remotely and it seems to work. Thanks!
I'm not the original asker, but I've got a graph dataset generated/manipulated with python with about 1 million points and lines, and I just can't find a good interactive web visualization for it. D3, Leaflet, about 15 other things I looked at - the data set is just too big. Maybe kepler.gl works, but if you know off the top of you head a better way to viz this, please let me know!
interesting to see this released. Have used DeckGL previously for data viz (DeckGl is well done and easy to use if looking for a javascript/node dataviz toolset), so rather nice to see the spatial demos, packaged up and built out simplifying adoption for geospatial apps. very cool.
Yes, integrated into kepler.gl is the ability to add a custom Mapbox GL style created in Mapbox Studio. You can design a custom map, add custom data from a shapefile, geojson, or geoTIFF file, and pull it into your kepler.gl dashboard.
I'd love to chat with some of the Uber engineers who worked on this. I'm a Redux maintainer, and I'm always eager to hear feedback from people who are building serious applications with Redux.