Cozy

sketch > cozy

[18:51] <arcol_> I would like to ask, to have two servers (2 raspberry pis at different isp), which are in sync (cozy.mydomain.com, cozy2.mydomain.com), and if one is offline (ISP trouble), then I could use the second one, and no worry about syncing
[18:51] <arcol_> ehh. My question is, if it is possible
[18:52] <arcol_> or is it a one server scenario all the time. (be it on a VPS or on a raspberry pi)
[18:55] <newebe> It's possible
[18:55] <newebe> but you'll have to configure couchdb yourself
[18:55] <newebe> we don't manage it through cozy yet
[18:55] <newebe> to achieve that you can sync both couchdb databases
[18:55] <newebe> with the continuous replication feature
[18:56] <newebe> hi fflorent
[18:56] <fflorent> newebe: Hi !
[18:56] <newebe> how are you doing?
[18:56] <fflorent> fine and you ?
[18:56] <newebe> good
[18:57] <newebe> I published two standalone apps
[18:57] <fflorent> I will attend a conference in 5 minutes ;).
[18:57] <fflorent> May I help you somehow?
[18:57] <newebe> which conference?
[18:57] <newebe> no help needed
[18:58] <newebe> I was just thinking that our standalone apps might interest you
[18:58] <arcol_> right now, I'm using bittorrent sync (for file sharing). I'm using it on 4 devices, two of it are raspberry pis. I connected an 1TB external hdd, and I'm using 4GB swapfile (on the hdd) to solve memory problems. Everything works fine. But I need to develop on my own a file indexer, and some ajax web interface, but discovered cozy just today.
[18:58] <arcol_> with bittorrent sync I didn't have to worry about dynamic ips (all of my 4 devices are on dynamic ips)
[18:58] <fflorent> newebe: Ah ?
[18:59] <newebe> fflorent: it's based on pouchdb and you can setup them
[18:59] <newebe> just by typing npm install cozy-calendar -g
[18:59] <newebe> or npm install cozy-tasky -g
[18:59] <newebe> and the environment development requires nothing more than node
[18:59] <newebe> arcol_: Cozy is not as advanced as bittorent
[19:00] <newebe> but yes it comes with pretty interface and many more features
[19:01] <arcol_> basically what I'm after is reliable internet connection without renting a VPS. With 3 locations, its basically 100% reliable (I'm testing it since this february).
[19:02] <fflorent> newebe: Right. Currently I am not a really Cozy user because I don't have a 24/24 server running it (I am just curious about it), and I am not a Cozy developper except for some patches I submitted ;)
[19:02] <fflorent> But thanks for the information. I may be interested in some days when I have my own server
[19:03] <arcol_> because I don't have a 24/24 server running it <— this. put it on two raspberry pis, one at home, one at your parent's home, and you are done.:)
[19:03] <newebe> fflorent: ok you can run the apps on your desktop too
[19:03] <arcol_> it is basically 150-200EUR investment depending on the hard drive.
[19:04] <arcol_> fflorent: that is what I'm after
[19:04] <fflorent> ah, OK !
[19:04] <newebe> arcol_: standalone apps could interest you they are much lighter to run
[19:04] <newebe> arcol_: but synchronisation is not included :)
[19:05] <fflorent> arcol_: I plan to buy one (I just have a one with 256M of RAM)
[19:05] <arcol_> fflorent: ram is not an issue, you put a 4GB swapfile to the external hdd. The raspberry pi slow enough, to not notice swapfile being on a hdd…
[19:06] == NewCozyUser [~ten.hvo.4699822sn|sUyzoCweN#ten.hvo.4699822sn|sUyzoCweN] has joined #cozycloud
[19:07] <arcol_> fflorent: here is my notes: http://wiki.arcol.hu/sketch:rpi I tend to keep updating. I want to fit cozy into this architecture somehow
[19:07] == NewCozyUser [~ten.hvo.4699822sn|sUyzoCweN#ten.hvo.4699822sn|sUyzoCweN] has quit [Client Quit]
[19:08] <arcol_> basically a home internet is not reliable enough to be a single source. But at home I can easily attach 1TB storage room, which is really costly if you want to have it on a VPS or amazon
[19:08] <arcol_> storage
[19:27] <newebe> arcol_: thank you for the swap info for the raspberry pi
[19:27] <newebe> because cozy consomes a little bit too much RAM for the Pi
[19:28] <newebe> cozy consums
[19:30] <arcol_> newebe: there are still some bugs to kick out
[19:31] <arcol_> for example if you plug the external hdd too soon during boot, and /media/btsync exists, it will mounted to /media/btsync_ while the swapfile will be created to /media/btsync/swapfile, which is still on the root
[19:31] <arcol_> so basically you are out of space
[19:32] <arcol_> also if you unplug the external hdd while running, linux cant release the swapfile, it will stuck until reboot, but if you replug it reattach the new swapfile, and the linux prioritize between the swapfiles, so until you fill up the 4GB, you won't try to use the unavaible swapfile from the previous disk mount
[19:32] <arcol_> thats two bugs I noticed
[19:33] <arcol_> (long term usage)
[19:34] <arcol_> maybe cozy could be prepared for file syncing application. I'm bittorrent sync, but git-annex-assistant looks really promising
[19:34] <arcol_> I'm -> I have
[19:38] <jsilvestre> we're working on it, currently we use couchdb to store meta data and binaries but we are planning on using something else for binary sync
[19:39] <jsilvestre> see https://forum.cozy.io/t/syncing-binaries-in-cozy/31/4 :)
[19:43] <arcol_> jsilvestre: then I would onnly like to mention two additional candidates, just like what the last responder did on the forum: btsync, git-annex-assistant
[19:44] <jsilvestre> noted, thanks :)
[19:45] <jsilvestre> we discarded them when we first give it a try because couchdb is very convenient (i.e. no additional port to open and we already use it as our store engine) but they are probably worth another look
[19:46] <arcol_> with btsync I didn't had to mess with port opening on any router
[19:46] <arcol_> and it is syncing at home, parent's home, workplace, another workplace, mcdonald's, even at star bucks I believe
[19:47] <arcol_> but btsync is closed source
[19:47] <jsilvestre> you still
[19:47] <jsilvestre> need a port*
[19:47] <arcol_> i know
[19:47] <jsilvestre> we can't afford that
[19:47] <arcol_> it can be random
[19:47] <jsilvestre> at least without consideration
[19:48] <arcol_> also btsync autodiscovers others on the internet
[19:48] <arcol_> so no need fix ips
[19:48] <jsilvestre> I don't know the exact details, the whys but it appeared to be a problem with us hosting cozy opening multiple ports
[19:50] <jsilvestre> but we'll definitely look at those solutions anyway
[19:50] <jsilvestre> we haven't started working on that just yet
[19:51] <arcol_> the forum on the cozy website looks awesome. What software is it? Some builtin cozy app? (would be awesome and great marketing)
[19:55] <arcol_> if another port open is an issue you can always fall back on builtin solution. But play nice with others having different syncing solution would be a desirable thing. The only thing which needs to be solved is do not sync database files while writes happening
[19:55] <jsilvestre> nop it's discourse
[19:56] <jsilvestre> https://meta.discourse.org/
[19:56] <arcol_> I will definietly look closer to this cozy. If mail app has labelling features as gmail, it is a selling point for me
[19:56] <arcol_> I will make a backup email for sure using cozy
[19:56] <jsilvestre> we are in the process of building it
[19:57] <jsilvestre> I am in charge, actually ;-)
[19:57] <jsilvestre> our trello boards are public and I regularly release stuff https://trello.com/b/jY0qR3lH/emails
[19:57] <jsilvestre> it's only client stuff, there is no imap binding yet
[19:58] <arcol_> well, can I share my two cents? If I don't fatigue you
[19:59] <jsilvestre> yes please
[20:01] <arcol_> 1. use one file per email. Be it mailbox or any other custom format
[20:01] <arcol_> 2. extract attachment all the time, and store it separately. Replace it with hyperlinks in the email
[20:01] <jsilvestre> i don't get your 1.
[20:02] <jsilvestre> about 2. we are going to use the same binary system we use for our "dropbox" app
[20:02] <jsilvestre> so I guess it's going to be like you said
[20:02] <arcol_> I don't think it is wise to store all the email in a big giant database
[20:02] <arcol_> my gmail box is 12GB
[20:02] <jsilvestre> I get it
[20:02] <jsilvestre> well couchdb is supposed to be able to handle many documents
[20:03] <jsilvestre> that's one of the reason we picked it
[20:03] <jsilvestre> but I need to make proper tests to figures if at least the interface can handle that much emails x)
[20:03] <arcol_> when you receive the emails auto tag them (extract contact, email address, attachment name, date), so when I search for email you only need to search in your tag database
[20:04] <jsilvestre> we are using an indexer (whoosh) so I guess that's okay too
[20:04] <jsilvestre> (I'm still noting your precious advices, thanks!)
[20:04] <arcol_> also have a way to backup emails (all sent and received emails stored at a different place too. It is desirable for companies, who want to save verbatim copies of workers email sending. So it is some kind of proof)
[20:05] <arcol_> but only one time, when the sending or receiving happens Its kind of redirecting email
[20:05] <jsilvestre> not sure it's relevant to a personal approach
[20:05] <arcol_> also have a way of archiving emails. At workplace, there are people who have 40GB of emailbox. Basically they are sending huge attachments around all the time
[20:06] <arcol_> builtin send attachment as url, so don't send 20MB email as an attachment, but save the attachment in cozy, and send a hyperlink to that file
[20:07] <jsilvestre> that is smart, good point
[20:07] <arcol_> I don't think a general indexer would solve all the problem. It will become quickly a huge bottleneck, when 90% of the time you only want to search to title, email address and name
[20:07] <arcol_> so indexing each and every email is unnecessary
[20:08] <jsilvestre> i get your point
[20:08] <arcol_> tagging (labelling) email is absolute must (maybe just my workflow, but I almost tag every email myself)
[20:08] <arcol_> I even use them when I search
[20:09] <arcol_> also gmail does not index everything, it index like 6months back everything and earlier message only the sender and the title or so
[20:09] <arcol_> try to search for older (2-3 years old) message, it is really hard
[20:10] <jsilvestre> we are fond of tagging
[20:10] <jsilvestre> I developed a task manager based on tags
[20:10] <arcol_> also super simple installation of an email server for 10-15 people top (family, small business)
[20:10] <jsilvestre> so be sure you will be able to add plenty of them :D
[20:11] <jsilvestre> we don't plan on adding email server just yet….
[20:11] <arcol_> if you look installing a postfix, imap server, webdav, a *BACKUP server*, spam filter is super hard
[20:11] <jsilvestre> it's going to be a big topic for us in the next few months
[20:11] <arcol_> takes a week minimum if you are doing the first time
[20:12] <arcol_> and most of the time people want 10-20 email address top, and don't need all the complication
[20:12] <arcol_> basically really standard setup
[20:12] <arcol_> if you could replace that basic need, you would basically stop gmail migration (many small business migrates over gmail, because its hassle-free)
[20:12] <jsilvestre> yeah we know that, that's why we don't want to hurry up
[20:13] <jsilvestre> so for now, we'll just be providing a client
[20:13] <jsilvestre> but since we do want to release ppl from gmail too, we'll have to do something at some point
[20:13] <arcol_> but really plan ahead for big data. Like 100GB per user. But basically it is 90GB attachment and 10Gb email or even less
[20:13] <arcol_> so don't put everything in databases, it will just consumes twice the place it needs
[20:14] <arcol_> and searching in mails should be priority #1
[20:14] <arcol_> gmail is really good at this, I know companies who redirects emails to gmail (as backup solution), and also for easy searching
[20:15] <jsilvestre> there are 2 things we value a lot for this app: finding what you are looking for AND having interactions with the rest of your data (i.e. other cozy apps)
[20:15] <arcol_> I need to run, thanks for listening. I will definietly give a spin to this cozy thing. Looks promising, and its trendy
[20:15] <jsilvestre> thank you very much for the advice!
[20:15] <jsilvestre> feel free to ask more questions if you find trouble getting started
[20:16] <arcol_> I would really like to be part of mail application development, with feedbacks at least. I have used many mail programs, backed darkmail, looked at protonmail.ch, so it is always on the back of my mind

Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-NonCommercial-NoDerivs 3.0 License