Hacker News new | past | comments | ask | show | jobs | submit login
Show HN: I Built a Metasearch Engine with React, Redux, Express and TypeScript (github.com/joshuascript)
79 points by _yljl on Oct 15, 2018 | hide | past | favorite | 42 comments



This is a neat project.

I would encourage more development be done to have the type cover more of the front end code. Your development environment must not be of much help due to the heavy use of the "any" type throughout the codebase, which essentially escapes out of the type system.


Thanks, it started out as regular, valid JS, then I figured I might as well convert it to TypeScript, and I did so (perhaps too) quickly. I do plan to add stronger typing later.


What advantages did you get from typescript? Seems like type safety is a huge trade off from what you lost by adding those amounts of complexity, especially when you’re not really utilising it.


You'd get more benefit if you're developing API where random others are consuming your code for them to use API properly.

But still, it can still save you from obvious mistakes even for a one man project.


I agree but I’d pick graphql for that instead of typescript.


Not sure TypeScript and GraphQL are mutually exclusive.


They aren’t, but graphql would give you safer types in your api than typescript.


Not sure one gives you safer typing than the other in absolute. They both have a different context/purpose. TypeScript is great for typing a code base for function calls type of APIs, whereas GraphQL is great to express types for inter-system communication (usually over HTTP).

Using GraphQL to type your in-process API would be overkilled and would require quite a bit of contortionism, and while you can share same TypeScript types between your end-points interfaces (and can be a good first step), inter-process communication is better expressed with something like GraphQL or Protobuf.


Honestly, I'm using it for it's marketability. I haven't yet found it particularly useful.


I see that you are scraping the results directly, curious if you’d tried out Google’s json search api? Any thoughts about it? I was about to need it or similar for a project.


Google's json search api gives you a consistent easy to parse api.

However it requires a Google API key, which limits free usage.

Client-side scraping scales effectively, as it distributes the reads onto all the clients. But it's also more brittle, as Google could change a small implementation of their search and break all the scraping functions.


I've tried it with the C# wrapper library, so using plain old methods and C# objects rather than touching the JSON directly.

It's not too bad, but not free as far as I'm aware and the result snippets they give you are quite limited in length. Also I found it a little tricky to get good results on some queries when I tried it out.


I haven't tried it, I'm afraid.


Nice to see a project using React, Typescript and Redux in the wild.


Believe me, there are plenty of companies using that trio in production.


[flagged]


Well, typescript is nice. But. The amount of boilerplate I write with redux is downright annoying, add typescript to the mix and the interfaces, typedefinitions, enums, various scattered exports, actioncreators and typehelpers are just mind-boggling - I don't like it.

Heck, I already have huge livetemplates in my ide to generate most of these stuff, but it feels wrong. And while we have fun writing "enterprise quality code", routing falls apart, load indicators are missing, form validators are a mess, error handling is broken, etc.


Hi, I'm a Redux maintainer.

We're currently working on a package called `redux-starter-kit` that contains some utilities to help simplify common concerns about Redux, including store setup, generating action creators, and writing reducer logic and immutable updates:

https://github.com/reduxjs/redux-starter-kit

I'd appreciate it if you could take a look, try it out, and give us some feedback.

It's currently JS-only, but we're planning to convert it to be written in TS in the near future.

As for as Redux and TS in general... I know I've seen comments similar to yours ("duplicating action constants and enums", etc). Unfortunately, I don't use TS myself, and don't know enough to offer meaningful suggestions there.

If you've got specific pain points with Redux and TS, please go ahead and file an issue for discussion, and perhaps we can work on some solutions.


I recently started looking at mobx-state-tree[1] and I feel like it solves a lot of the boilerplate needed with redux in a very nice opinionated way. In particular it has built in support for tracking action history (as list of patches and snapshot), building views of computed values (selectors), organizing async action flows, and colocating all of this logic in a single model class.

It can be used on top of redux but it won't really look like redux code anymore. But since you're using immer in the starter, it wasn't going to look like redux anyway (e.g., ton of triple dots everywhere).

Do you have any opinion on mobx-state-tree? Maybe you should just bless mobx-state-tree in the starter too? (immer is built by the same mobx guy)

[1] https://github.com/mobxjs/mobx-state-tree

(I have not actually use the library yet. Secretly hoping you'll will tell me about all the cons so I don't have to do the research myself. Currently choosing a library based on which has the best saga-like support)


mobx-state-tree (mst) is an addition to mobx. While mobx is a rather generic approach to manage state via composable observables and actions, mst is a strongly opinionated enhancement of it that brings in e.g. strong typing and some simplifications. While mst was great for some simple things, it soon runs into issues at complex requirements, one of them being performance. In my scenarios after running into some of these issues, I then rebuilt everything in mobx and brought in the strong typing etc. via TypeScript and code governance. This worked great so far, not much boilerplate, great state/action componentisation, also a great serialization/deserialization out of the box with serializr.


Hi, I've not actually used redux at any scale but I have used react a lot (sounds silly?).

I was wondering, would it be possible to have something like a strict subset of redux with minimal boilerplate and very few and very simple API functions which are redux compatible. But which can eventually be swapped with the full Redux library if need be?


No, not really. Redux itself _is_ already small - if you strip out the safety checks, it's less than 100 lines [0], and you can see the core concept in less than 25 [1].

Part of the issue is that people use the word "boilerplate" to refer to many different things, such as writing action creator functions, writing immutable update logic, calling `connect()` to generate wrapper components that talk to the Redux store, etc. Everyone seems to have a different thing that they are concerned about.

The "starter kit" package I linked tries to address several of the most common concerns, while still keeping Redux the way it's meant to be used.

[0] https://gist.github.com/gaearon/ffd88b0e4f00b22c3159

[1] https://blog.isquaredsoftware.com/2017/05/idiomatic-redux-ta...


Thank you! I've been thinking of writing a cli helper to generate the boilerplate, but that would be specific to the project I'm working on.

I really loved working with angular-cli, where you can create most of the skeletons with the cli tool (which in turn is supported by webstorm).


I know this is going to sound like blasphemy, but is the TS version also going to support Angular?


You can already use Redux with any UI layer or framework, whether it be React, Angular, Vue, jQuery, or vanilla JS.

The starter kit package just provides some additional utilities for common use cases. You could use it in a TS app already, you'd just have to stub out the type declarations yourself. Reworking the package to be written in TS is mostly to allow easier integration with users building their apps in TS.


isn't ngrx the popular store implementation for angular projects?


Pretty much. I'm using ngxs in my ng6 project at work and it'a fantasic library. Way less boilerplate than Redux w/TS.


This is why we are using MobX at work. The amount of boilerplate code in Redux is insane.


I would just love to work with mobx, coming from vue, the project is already big enough and a rewrite is not an option in this stage. Are people using redux and mobx together?


I'm a fan of mobx, using Typescript also.


You could try the https://github.com/Hotell/rex-tils package. I find these action creator utilities make typing the redux concepts a little easier.

Also, I am working on a library which aims to centralize all of the redux constructs into reusable modules. It doesn't eliminate all if the boilerplate, but I think it makes it easier to add things to the store. https://redux-dynamic-modules.js.org


Check out ReasonML with reason-react. Immutability, reducers and type inference built in. Way less code to write and strong guarantees by the compiler.


I've really enjoyed using rematch[0] on top of redux to reduce boilerplate.

0: https://rematch.gitbooks.io/rematch/


State management can be simplified using streams & the meiosis pattern. No real need for redux/mobx/etc


That' true. But wherever I look, projects and people bought into this idea that if it's good enough for Facebook (or Google in the case of Angular) then going with react+redux is a future-proof idea.


This is cool...


You mean you built a frontend for Google, not a search engine.


That's literally in the title. It says METASearch engine, not search engine. I have never claimed it was a search engine.

https://en.wikipedia.org/wiki/Metasearch_engine


OP is being bit dismissive, but IMHO the key feature of a metasearch engine is the processing of results, not the interface. Filtering Google’s results is technically a metasearch engine... but it’s a fairly trivial example.


https://github.com/JoshuaScript/spresso-search/blob/master/s... - this will be blocked by Google if used at any kind of scale.


I was looking into how google was implemented initially with the pagerank algorithm and I think I am getting closer. Do they still use the same algorithm? I ahve no concept of how things behave when they scale or how to scale for that matter.


I think they just use BrainRank for search now. It still incorporates the basic premise of pagerank, but the rules are not hand written anymore, it's all done with Tensorflow.


Well now they're baking in things like piracy and fake news into the mix and I've noticed a loss in quality for regular search results over the years.




Consider applying for YC's W25 batch! Applications are open till Nov 12.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: