markwylde

twtxt.net

An developer addicted to JavaScript

Recent twts from markwylde
In-reply-to » Oh farrrrk me! 🤦‍♂️ React Apps, NodeJS c'mon! 🤦‍♂️ How does anyone do this shit?! 🤯

The problem isn’t with NodeJS or NPM, it’s developers that are so willing to use horrible frameworks/libraries/tooling that is just simply not needed. NodeJS gives you so much out of the box, and NPM is simply a place to store your packages. With Deno, you won’t even need a package manager as it takes a step closer to go modules approach.

If you’re going to use React, TypeScript, NextJs, Webpack, Styled Components, Material UI, Jest, and the 10k dependencies that comes with it, then yeah, your dev environment is going to be slow, bloated, and incredibly frustrating to work with. Not to mention have security issues. I’ve literally just done a fresh create-react-app installation (latest version 5.0.1 as I write this), and boom, 6 high severity vulnerabilities.

Image

But it doesn’t have to be this way. Choose lightweight libraries that do one thing really well, and build your project from the ground up yourself.

⤋ Read More
In-reply-to » Oh farrrrk me! 🤦‍♂️ React Apps, NodeJS c'mon! 🤦‍♂️ How does anyone do this shit?! 🤯

The problem isn’t with NodeJS or NPM, it’s the developers that are so willing to use horrible frameworks/libraries/tooling that is just simply not needed. NodeJS gives you so much out of the box, and NPM is simply a place to store your packages. With Deno, you won’t even need a package manager as it takes a step closer to go modules approach.

If you’re going to use React, TypeScript, NextJs, Webpack, Styled Components, Material UI, Jest, and the 10k dependencies that comes with it, then yeah, your dev environment is going to be slow, bloated, and incredibly frustrating to work with. Not to mention have security issues. I’ve literally just done a fresh create-react-app installation (latest version 5.0.1 as I write this), and boom, 6 high severity vulnerabilities.

Image

But it doesn’t have to be this way. Choose lightweight libraries that do one thing really well, and build your project from the ground up yourself.

⤋ Read More
In-reply-to » How come there is only 25 pages of twts on the feed?

@prologic@twtxt.net for example, looking at your feed it says your first post was at 2022-03-09T23:38:24Z. But I know you were posting way before that. But if I look at mine, I see my first post was 2020-09-30T12:50:42Z.

So maybe it’s cut off by file size?

⤋ Read More
In-reply-to » I love simple, lightweight, small, minimal tools that just do the bare minimum. Based on that, does anyone have any good recommendations for a key value store that is: - lightweight - clustered - sharded (so if I have 5 instances and 100 keys, each node will roughly have 20 keys on it). - easy to join nodes: as in kv-server --join somehost:1111

Did a quick benchmark:
https://git.mills.io/prologic/bitraft/issues/58

Seems the summary benchmark of a 5node cluster on my laptop is:

GET: 1165.64 requests per second
SET: 1061.80 requests per second

⤋ Read More
In-reply-to » I love simple, lightweight, small, minimal tools that just do the bare minimum. Based on that, does anyone have any good recommendations for a key value store that is: - lightweight - clustered - sharded (so if I have 5 instances and 100 keys, each node will roughly have 20 keys on it). - easy to join nodes: as in kv-server --join somehost:1111

But in Bitraft every node contains every key + value, right? I probably wasn’t clear above, but in my idea REPLICA_COUNT would be 3 but the NODE_COUNT may be 10. So a put would go to 3 of 10 of the nodes.

⤋ Read More
In-reply-to » I love simple, lightweight, small, minimal tools that just do the bare minimum. Based on that, does anyone have any good recommendations for a key value store that is: - lightweight - clustered - sharded (so if I have 5 instances and 100 keys, each node will roughly have 20 keys on it). - easy to join nodes: as in kv-server --join somehost:1111

@prologic@twtxt.net I’m happy to do it. Might try now actually. It was just incase you knew. I’ll post in the README if I get it working. I’m hoping redis-benchmark will work since it’s got the same api as redis.

I wonder if sharding could be implemented by:

Presumptions:

  • redis can broadcast to all nodes in the cluster
  • REPLICA_COUNT is 3

PUT workflow:

  • a PUT get’s forwarded to REPLICA_COUNT random nodes in the cluster

GET workflow:

  • a broadcast is made to the cluster saying “I NEED A VALUE FOR KEY ‘TEST’”
  • all nodes that contain that value reply to the server
  • the first response get’s forwarded to the client
  • the other responses are discarded

I’m sure there would be some edges cases, like syncing.

  • What if 1 of the random node’s is full and therefore only REPLICA_COUNT-1 nodes received the document
  • This could me 2 nodes have the new value, but the 3rd has the old value

Maybe it could be solved by only committing once REPLICA_COUNT nodes successfully receive the message.

⤋ Read More
In-reply-to » I love simple, lightweight, small, minimal tools that just do the bare minimum. Based on that, does anyone have any good recommendations for a key value store that is: - lightweight - clustered - sharded (so if I have 5 instances and 100 keys, each node will roughly have 20 keys on it). - easy to join nodes: as in kv-server --join somehost:1111

@prologic@twtxt.net ever done any stress testing on bitraft? In a cluster, do you know that the throughput would be? Like, PUT’s per second and GET’s per second?

⤋ Read More
In-reply-to » I love simple, lightweight, small, minimal tools that just do the bare minimum. Based on that, does anyone have any good recommendations for a key value store that is: - lightweight - clustered - sharded (so if I have 5 instances and 100 keys, each node will roughly have 20 keys on it). - easy to join nodes: as in kv-server --join somehost:1111

Thanks guys. Bitraft is awesome @prologic@twtxt.net but yeah, not sharded :( I did try etcd before @abucci@anthony.buc.ci but I did find it tricker to setup than Bitraft. But again, it’s not sharded :(

⤋ Read More

I love simple, lightweight, small, minimal tools that just do the bare minimum. Based on that, does anyone have any good recommendations for a key value store that is:

  • lightweight
  • clustered
  • sharded (so if I have 5 instances and 100 keys, each node will roughly have 20 keys on it).
  • easy to join nodes: as in kv-server --join somehost:1111

For reference, I think Consul is too heavy (and not sharded I believe).

It would be great to have a small go executable, that I can run on 10 servers, all connected up, that exposes a redis like api. Simple GET, PUT and STREAM would be great.

@prologic@twtxt.net anyone else here I can ping?

⤋ Read More
In-reply-to » I'm looking into building a standalone app that can connect to any twtxt pod.

I think maybe an account swither in the nav would be a good balance to start with. If it turns out many people use multiple pods, we could improve the UI to make switching easier.

⤋ Read More
In-reply-to » I'm looking into building a standalone app that can connect to any twtxt pod.

@prologic@twtxt.net Yeah, this is what I’m thinking. How about, since the app would technically be able to connect to any pod, it could store a list of pods you’re a member of. For redundancy, for example.

So you would add an account (minimum of one), then you could also add additional. Then if you can’t connect to the first pod, you can connect to the second?

Still pointless? I mean. It’s easier to just have 1 account that’s stored and connected to every time. But I’m just wondering if anyone here does have multiple accounts.

⤋ Read More

I’m looking into building a standalone app that can connect to any twtxt pod.

I’m considering a slack like approach, where down the left hand side you can have multiple pod instances.

Do people have an account on multiple pods? Or is there not really any point since you can feed into any pod from any other pod?

⤋ Read More