File Storage (Dropbox, Google Drive)

{ "type": "excalidraw", "version": 2, "source": "https://marketplace.visualstudio.com/items?itemName=pomdtr.excalidraw-editor", "elements": [ { "type": "text", "version": 669, "versionNonce": 389393430, "isDeleted": false, "id": "XWdf_RoKUph93_TGTlhN0", "fillStyle": "cross-hatch", "strokeWidth": 1, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "angle": 0, "x": 336.2136878027392, "y": 42.99524319033378, "strokeColor": "#000000", "backgroundColor": "#fff", "width": 518.4718017578125, "height": 92, "seed": 1291993419, "groupIds": [], "frameId": null, "roundness": null, "boundElements": [], "updated": 1767418956330, "link": null, "locked": false, "fontSize": 36, "fontFamily": 1, "text": "Design Dropbox, Google Drive\n", "textAlign": "left", "verticalAlign": "top", "containerId": null, "originalText": "Design Dropbox, Google Drive\n", "lineHeight": 1.2777777777777777, "index": "a0", "autoResize": true }, { "type": "text", "version": 2150, "versionNonce": 167941514, "isDeleted": false, "id": "jNghba2gtFWznv5aAC3i8", "fillStyle": "solid", "strokeWidth": 1, "strokeStyle": "solid", "roughness": 0, "opacity": 100, "angle": 0, "x": 355.1426952486048, "y": 190.42343224345075, "strokeColor": "#1e1e1e", "backgroundColor": "#fefefe", "width": 654.359375, "height": 300, "seed": 1982369963, "groupIds": [], "frameId": null, "roundness": null, "boundElements": [], "updated": 1767419251619, "link": null, "locked": false, "fontSize": 20, "fontFamily": 1, "text": "Functional Req\n\n- upload a file (Users can create and upload files)\n- download a file\n- automaticall sync files across devices (Changes to files should \nbe propagated to all devices will access)\n- Files can be shared across multiple \ndifferent users\n- We should be able to see older versions\nof files\n\n# of Reads = # of Writes (roughly)", "textAlign": "left", "verticalAlign": "top", "containerId": null, "originalText": "Functional Req\n\n- upload a file (Users can create and upload files)\n- download a file\n- automaticall sync files across devices (Changes to files should \nbe propagated to all devices will access)\n- Files can be shared across multiple \ndifferent users\n- We should be able to see older versions\nof files\n\n# of Reads = # of Writes (roughly)", "lineHeight": 1.25, "index": "a4", "autoResize": true }, { "type": "text", "version": 5627, "versionNonce": 1045992534, "isDeleted": false, "id": "OmKQPstnDQbKxMIXYM0nK", "fillStyle": "hachure", "strokeWidth": 1, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "angle": 0, "x": 1762.8326481830545, "y": 206.26250588842345, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "width": 429.8396911621094, "height": 234, "seed": 528157925, "groupIds": [], "frameId": null, "roundness": null, "boundElements": [], "updated": 1767419382660, "link": null, "locked": false, "fontSize": 20, "fontFamily": 1, "text": "Back of Envelope\n\n- 1 billion users\n- about 100 docs per user\n- each doc about 1kb, some in megabytes\n- 1 billion * 100 * 100kb = 100 TB of docs\n- Most docs shared with < 10, some shared\nwith many thousands\n-> import when propagating changes", "textAlign": "left", "verticalAlign": "top", "containerId": null, "originalText": "Back of Envelope\n\n- 1 billion users\n- about 100 docs per user\n- each doc about 1kb, some in megabytes\n- 1 billion * 100 * 100kb = 100 TB of docs\n- Most docs shared with < 10, some shared\nwith many thousands\n-> import when propagating changes", "lineHeight": 1.3, "index": "a6", "autoResize": true }, { "type": "text", "version": 5066, "versionNonce": 328375894, "isDeleted": false, "id": "uTACGkxlMpJt8JhtT4DCe", "fillStyle": "solid", "strokeWidth": 1, "strokeStyle": "solid", "roughness": 0, "opacity": 100, "angle": 0, "x": 354.74760991826724, "y": 606.4523977177129, "strokeColor": "#1e1e1e", "backgroundColor": "#fefefe", "width": 756.5994873046875, "height": 1352, "seed": 1694456107, "groupIds": [], "frameId": null, "roundness": null, "boundElements": [], "updated": 1767419099517, "link": null, "locked": false, "fontSize": 20, "fontFamily": 1, "text": "High Level Design\n\nDocument Permissions\n\nSchema\n\nuserId | docId\n\nIndexing and partitioning by userId allows us to quickly see\nwhich documents a given user has access to!\n-> useful on client startup\n\nReplication?\n-> Single leader may be fest for adds and subsequent removes\n-> A CRDT could fix this (see add-remove set CRDT)\n-> Probably overkill, this table unlikely to be a big bottleneck\n-> Let's keep things simple and use MySQL\n\nFile Uploads\n\nSplit files into smaller chunks!\n\nPros:\n- Parallel uploading to S3\n- Only load modified chunks on file changes\n\nCons:\n- Added complexity on client to do file diffing, have to keep track on chunks\nin DB!\n\nChunking data schema\n\nChunks table:\nID | Version | Hash | Link (could use wide column/document format)\n\nIndexing by Id then version allows quick queries to determine all chunks for\na given file version.\n\nfileUploadApi {\n uploadType\n data\n}\n\nfileDownloadApi {\n filePath\n}\n\nfileHistoryApi {\n filePath\n limitOfRevisions\n}\n", "textAlign": "left", "verticalAlign": "top", "containerId": null, "originalText": "High Level Design\n\nDocument Permissions\n\nSchema\n\nuserId | docId\n\nIndexing and partitioning by userId allows us to quickly see\nwhich documents a given user has access to!\n-> useful on client startup\n\nReplication?\n-> Single leader may be fest for adds and subsequent removes\n-> A CRDT could fix this (see add-remove set CRDT)\n-> Probably overkill, this table unlikely to be a big bottleneck\n-> Let's keep things simple and use MySQL\n\nFile Uploads\n\nSplit files into smaller chunks!\n\nPros:\n- Parallel uploading to S3\n- Only load modified chunks on file changes\n\nCons:\n- Added complexity on client to do file diffing, have to keep track on chunks\nin DB!\n\nChunking data schema\n\nChunks table:\nID | Version | Hash | Link (could use wide column/document format)\n\nIndexing by Id then version allows quick queries to determine all chunks for\na given file version.\n\nfileUploadApi {\n uploadType\n data\n}\n\nfileDownloadApi {\n filePath\n}\n\nfileHistoryApi {\n filePath\n limitOfRevisions\n}\n", "lineHeight": 1.3, "index": "aF", "autoResize": true }, { "type": "ellipse", "version": 2416, "versionNonce": 321161674, "isDeleted": false, "id": "OHvz-4zhuR0M6MCP8kSLd", "fillStyle": "solid", "strokeWidth": 1, "strokeStyle": "solid", "roughness": 2, "opacity": 100, "angle": 0, "x": 3642.0315066083876, "y": 743.6596898924304, "strokeColor": "transparent", "backgroundColor": "#fff", "width": 117.47426036349037, "height": 119.95526047648993, "seed": 916377701, "groupIds": [ "uyY0z8cKaYyye2rZ3VNPo", "uCeitmaYnpbaebK8vufU9", "6C3_LvfRD44vdBoCK3Dng" ], "frameId": null, "roundness": { "type": 2 }, "boundElements": [], "updated": 1767419564120, "link": null, "locked": false, "index": "ah" }, { "type": "ellipse", "version": 2007, "versionNonce": 1668143126, "isDeleted": false, "id": "ZKHtfjRND0MAQt1UjvFzi", "fillStyle": "cross-hatch", "strokeWidth": 1, "strokeStyle": "dashed", "roughness": 2, "opacity": 100, "angle": 0, "x": 3641.0730069349165, "y": 744.1054226530644, "strokeColor": "#000000", "backgroundColor": "#ced4da", "width": 117.47426036349037, "height": 119.95526047648993, "seed": 1704033221, "groupIds": [ "6C3_LvfRD44vdBoCK3Dng" ], "frameId": null, "roundness": { "type": 2 }, "boundElements": [ { "id": "MSLmEtYuzXek5T7iMK5be", "type": "arrow" }, { "id": "pw8kRWNP25SLmtcK8nAU2", "type": "arrow" }, { "id": "Rc8YBN087bJ_bRhrIGn2B", "type": "arrow" } ], "updated": 1767420191558, "link": null, "locked": false, "index": "ai" }, { "type": "text", "version": 2097, "versionNonce": 661935306, "isDeleted": false, "id": "03dJXx6ojpMCF5rgUHllV", "fillStyle": "cross-hatch", "strokeWidth": 1, "strokeStyle": "dashed", "roughness": 2, "opacity": 100, "angle": 0, "x": 3676.7397534484617, "y": 791.0123914115693, "strokeColor": "#000000", "backgroundColor": "#ced4da", "width": 36.585357666015625, "height": 37.8915993367056, "seed": 735558437, "groupIds": [ "6C3_LvfRD44vdBoCK3Dng" ], "frameId": null, "roundness": { "type": 2 }, "boundElements": [], "updated": 1767419564120, "link": null, "locked": false, "fontSize": 29.896674320761043, "fontFamily": 1, "text": "s3", "textAlign": "center", "verticalAlign": "top", "containerId": null, "originalText": "s3", "lineHeight": 1.2674185406098053, "index": "aj", "autoResize": true }, { "type": "arrow", "version": 424, "versionNonce": 1929576394, "isDeleted": false, "id": "MSLmEtYuzXek5T7iMK5be", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 0, "opacity": 100, "angle": 0, "x": 3390.8244678181536, "y": 967.2290246882985, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "width": 242.99187867892488, "height": 152.28368748449725, "seed": 946236299, "groupIds": [], "frameId": null, "roundness": { "type": 2 }, "boundElements": [ { "type": "text", "id": "BVZhovGeK_S9So2-Xm0wi" } ], "updated": 1767419868485, "link": null, "locked": false, "startBinding": { "elementId": "uWSC4vhMFzkrAjqvSWrpH", "focus": 0.3040043186622115, "gap": 11.508220553309567 }, "endBinding": { "elementId": "ZKHtfjRND0MAQt1UjvFzi", "focus": 0.3150813821589289, "gap": 8.112735498900078 }, "lastCommittedPoint": null, "startArrowhead": null, "endArrowhead": "arrow", "points": [ [ 0, 0 ], [ 242.99187867892488, -152.28368748449725 ] ], "index": "ak" }, { "id": "BVZhovGeK_S9So2-Xm0wi", "type": "text", "x": 3417.310481315331, "y": 866.0871809460499, "width": 190.0198516845703, "height": 50, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "akV", "roundness": null, "seed": 1206922454, "version": 34, "versionNonce": 1255053142, "isDeleted": false, "boundElements": [], "updated": 1767419866972, "link": null, "locked": false, "text": "request a presigned\nurl", "fontSize": 20, "fontFamily": 5, "textAlign": "center", "verticalAlign": "middle", "containerId": "MSLmEtYuzXek5T7iMK5be", "originalText": "request a presigned url", "autoResize": true, "lineHeight": 1.25 }, { "id": "DUifRzvGr92iPK7c6Lii7", "type": "text", "x": 1308.058942758838, "y": 503.5373566931204, "width": 766.8394775390625, "height": 1575, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b1c", "roundness": null, "seed": 1475979926, "version": 20, "versionNonce": 1470939018, "isDeleted": false, "boundElements": [], "updated": 1767419005708, "link": null, "locked": false, "text": "Chunks DB Considerations\n\nWithin a single leader, we need to be able to use locking to keep the proper\nversion number\n\nLocking approach wouldn't work for a multileader setup.\nIf using last write wins, one file version gets clobbered, no history\n\nIf extra write throughput we could still work using Siblings\n\nNow when we perform anti-entropy we store file v4.1 and v4.2\nNext user to load file sees both versions and must merge locally before upload\n\nLet's avoid this though, would be necessary if write throughput is needed\n\nSeems useful to use an ACID database so that all rows are updated\natomically\n-> Serializbity ensures that one version is finished writing before another\ncan publish\n-> If trying to write a version that already exists, DB rejects write\n\nCan use MySQL -> single leader replication + transactions\n-> increase write throughput by partition on fileId (hash range)\n\nUploading Actual File\n\nWe want to upload as few chunks as possible\n\nStep 1: Write to S3\nStep 2: write to Chunks DB\n\nif step 2 fails, we may have unnecessary data in S3\nCan have cleanup job to poll s3 and check if that chunk lives\nsomewhere in the DB\n\nPushing File Changes to Users\n\nWe want to ideally keep the number of connections maintained by each\nclient low. The best way to do this is to have a single node responsible\nfor a given client. Not great for popular documents\n\nFor most documents, < 10 users have access\n-> We can push these somewhere for the user ( shard by userId )\n\nFor super popular documents, 100k users have access\n-> Push changes to a popular document service (shard by document Id)\n\nFile Routing\n\nUpon making changes to a file, we need to propagate the changes to the\nright place!\n-> Similar to news feed design!\n\nUse CDC/Flink to ensure that we store a local copy of permissions and don't\nhave to query DB!\n\n\nMetadata Reading\n Since we will have many consumers of each metadata change, something\nlike Kafka makes sense here (uses long polling under the hood)\n-> Persistence ensures that clients that go down can re-read missed changes\n\n", "fontSize": 20, "fontFamily": 5, "textAlign": "left", "verticalAlign": "top", "containerId": null, "originalText": "Chunks DB Considerations\n\nWithin a single leader, we need to be able to use locking to keep the proper\nversion number\n\nLocking approach wouldn't work for a multileader setup.\nIf using last write wins, one file version gets clobbered, no history\n\nIf extra write throughput we could still work using Siblings\n\nNow when we perform anti-entropy we store file v4.1 and v4.2\nNext user to load file sees both versions and must merge locally before upload\n\nLet's avoid this though, would be necessary if write throughput is needed\n\nSeems useful to use an ACID database so that all rows are updated\natomically\n-> Serializbity ensures that one version is finished writing before another\ncan publish\n-> If trying to write a version that already exists, DB rejects write\n\nCan use MySQL -> single leader replication + transactions\n-> increase write throughput by partition on fileId (hash range)\n\nUploading Actual File\n\nWe want to upload as few chunks as possible\n\nStep 1: Write to S3\nStep 2: write to Chunks DB\n\nif step 2 fails, we may have unnecessary data in S3\nCan have cleanup job to poll s3 and check if that chunk lives\nsomewhere in the DB\n\nPushing File Changes to Users\n\nWe want to ideally keep the number of connections maintained by each\nclient low. The best way to do this is to have a single node responsible\nfor a given client. Not great for popular documents\n\nFor most documents, < 10 users have access\n-> We can push these somewhere for the user ( shard by userId )\n\nFor super popular documents, 100k users have access\n-> Push changes to a popular document service (shard by document Id)\n\nFile Routing\n\nUpon making changes to a file, we need to propagate the changes to the\nright place!\n-> Similar to news feed design!\n\nUse CDC/Flink to ensure that we store a local copy of permissions and don't\nhave to query DB!\n\n\nMetadata Reading\n Since we will have many consumers of each metadata change, something\nlike Kafka makes sense here (uses long polling under the hood)\n-> Persistence ensures that clients that go down can re-read missed changes\n\n", "autoResize": true, "lineHeight": 1.25 }, { "id": "fkdATwSVL5xkaFc7GKWj3", "type": "text", "x": 1096.425596615201, "y": 254.02192919429444, "width": 531.8196411132812, "height": 175, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b1d", "roundness": null, "seed": 1630765270, "version": 295, "versionNonce": 551662294, "isDeleted": false, "boundElements": [], "updated": 1767419405021, "link": null, "locked": false, "text": "Non Functional Req\n- Scale: 1 billion users (100 docs per users)\n- availability >> consistency\n- low latency upload and download (as low as possible)\n- support large files as 50 gb\n - resumable uploads\n- high data integrity (sync accuracy)", "fontSize": 20, "fontFamily": 5, "textAlign": "left", "verticalAlign": "top", "containerId": null, "originalText": "Non Functional Req\n- Scale: 1 billion users (100 docs per users)\n- availability >> consistency\n- low latency upload and download (as low as possible)\n- support large files as 50 gb\n - resumable uploads\n- high data integrity (sync accuracy)", "autoResize": true, "lineHeight": 1.25 }, { "id": "f31kR9gb_1U5gz_R21C6g", "type": "text", "x": -205.68090082251655, "y": 617.541799443441, "width": 166.33985900878906, "height": 100, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b1e", "roundness": null, "seed": 1301172886, "version": 58, "versionNonce": 28039818, "isDeleted": false, "boundElements": [], "updated": 1767419296266, "link": null, "locked": false, "text": "Core Entities\n- File (ray bytes)\n- File Metadata\n- Users", "fontSize": 20, "fontFamily": 5, "textAlign": "left", "verticalAlign": "top", "containerId": null, "originalText": "Core Entities\n- File (ray bytes)\n- File Metadata\n- Users", "autoResize": true, "lineHeight": 1.25 }, { "id": "NHmAmQr7MQk2lMiaUOckP", "type": "text", "x": -203.62140597215085, "y": 779.4670331736763, "width": 445.5196533203125, "height": 450, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b1f", "roundness": null, "seed": 1638547466, "version": 313, "versionNonce": 1047687574, "isDeleted": false, "boundElements": [], "updated": 1767421468085, "link": null, "locked": false, "text": "API\n\nPOST /files->200\nbody: File & FileMetadata\n\nGET /files/:fileId -> File & FileMetadata\n\nGET /changes?since={timestamp} -> fileIds[]\n\nPOST /files -> Presigned URL\nbody: FileMetadata\n\nPUT {Presigned URL}\nbody FileChunk\n\nPATCH /files -> 200\nbody: Partial <FileMetadat> \n(Chunk status updates)", "fontSize": 20, "fontFamily": 5, "textAlign": "left", "verticalAlign": "top", "containerId": null, "originalText": "API\n\nPOST /files->200\nbody: File & FileMetadata\n\nGET /files/:fileId -> File & FileMetadata\n\nGET /changes?since={timestamp} -> fileIds[]\n\nPOST /files -> Presigned URL\nbody: FileMetadata\n\nPUT {Presigned URL}\nbody FileChunk\n\nPATCH /files -> 200\nbody: Partial <FileMetadat> \n(Chunk status updates)", "autoResize": true, "lineHeight": 1.25 }, { "id": "BR2X28VQ55dV9-1aqPm9N", "type": "rectangle", "x": 2620.883347648523, "y": 899.3658175226552, "width": 227.33685664632412, "height": 537.3549568185506, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b1g", "roundness": { "type": 3 }, "seed": 1621535050, "version": 105, "versionNonce": 1521005462, "isDeleted": false, "boundElements": [ { "id": "kYcdFwMO2g18KBc5U3dzx", "type": "arrow" }, { "id": "FKgf2ItAmvz3Z33UsY-TC", "type": "arrow" }, { "id": "6oZcvla33sgDc669OjQQt", "type": "arrow" } ], "updated": 1767419767262, "link": null, "locked": false }, { "id": "rX7h2-Y5RvGJdgrtOgVrh", "type": "text", "x": 2653.375711333673, "y": 1088.859917513919, "width": 168.63986206054688, "height": 150, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b1h", "roundness": null, "seed": 40203786, "version": 111, "versionNonce": 1891813322, "isDeleted": false, "boundElements": [ { "id": "j9PmhfB6juRFVJH4csWhw", "type": "arrow" } ], "updated": 1768628643301, "link": null, "locked": false, "text": "API Gateway\nLoad Balance\n- authentication\n- rate limiting\n- SSL termination\n- Routing", "fontSize": 20, "fontFamily": 5, "textAlign": "left", "verticalAlign": "top", "containerId": null, "originalText": "API Gateway\nLoad Balance\n- authentication\n- rate limiting\n- SSL termination\n- Routing", "autoResize": true, "lineHeight": 1.25 }, { "id": "uWSC4vhMFzkrAjqvSWrpH", "type": "rectangle", "x": 3141.3705507760556, "y": 942.9245976467083, "width": 238.3667100353191, "height": 127.66598259637829, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b1i", "roundness": { "type": 3 }, "seed": 1714564362, "version": 91, "versionNonce": 1467716246, "isDeleted": false, "boundElements": [ { "type": "text", "id": "AIUhaI8_3ThddDhAVrb97" }, { "id": "MSLmEtYuzXek5T7iMK5be", "type": "arrow" }, { "id": "FKgf2ItAmvz3Z33UsY-TC", "type": "arrow" }, { "id": "323buhPH76iXg5AMYkwjo", "type": "arrow" }, { "id": "CgHtNwwVbxlsrM9x84VfC", "type": "arrow" }, { "id": "Rc8YBN087bJ_bRhrIGn2B", "type": "arrow" }, { "id": "3b757-G7rHCbnqi5oTG3-", "type": "arrow" }, { "id": "DgIi9RZCqHsba14sUi6kV", "type": "arrow" } ], "updated": 1768628647399, "link": null, "locked": false }, { "id": "AIUhaI8_3ThddDhAVrb97", "type": "text", "x": 3206.8639567580703, "y": 994.2575889448974, "width": 107.37989807128906, "height": 25, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b1j", "roundness": null, "seed": 1442181718, "version": 14, "versionNonce": 276169686, "isDeleted": false, "boundElements": [], "updated": 1767419559940, "link": null, "locked": false, "text": "File service", "fontSize": 20, "fontFamily": 5, "textAlign": "center", "verticalAlign": "middle", "containerId": "uWSC4vhMFzkrAjqvSWrpH", "originalText": "File service", "autoResize": true, "lineHeight": 1.25 }, { "id": "jhKgwi4-noczMtOqda8nw", "type": "ellipse", "x": 3663.4058319868454, "y": 1040.8940987646417, "width": 157.00463755529472, "height": 157.83444599255245, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b1k", "roundness": { "type": 2 }, "seed": 2046637334, "version": 39, "versionNonce": 1893167626, "isDeleted": false, "boundElements": [ { "type": "text", "id": "9v7WLR25Zn1KqNuIePTRv" }, { "id": "323buhPH76iXg5AMYkwjo", "type": "arrow" } ], "updated": 1767419652030, "link": null, "locked": false }, { "id": "9v7WLR25Zn1KqNuIePTRv", "type": "text", "x": 3692.938660251056, "y": 1070.0084182278401, "width": 97.91993713378906, "height": 100, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b1l", "roundness": null, "seed": 1770718794, "version": 31, "versionNonce": 405157066, "isDeleted": false, "boundElements": [], "updated": 1767420048358, "link": null, "locked": false, "text": "File\nmetadata\n(DynamoD\nB)", "fontSize": 20, "fontFamily": 5, "textAlign": "center", "verticalAlign": "middle", "containerId": "jhKgwi4-noczMtOqda8nw", "originalText": "File metadata\n(DynamoDB)", "autoResize": true, "lineHeight": 1.25 }, { "id": "WkYFicmgSMJ6F2rdw0g2V", "type": "text", "x": 3876.1662746867178, "y": 1011.753178938585, "width": 225.7598419189453, "height": 475, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b1m", "roundness": null, "seed": 596098378, "version": 241, "versionNonce": 2104065034, "isDeleted": false, "boundElements": [], "updated": 1767420280846, "link": null, "locked": false, "text": "File Metadata\n- FileID\n- File Name\n- MimeType\n- Compression Algo\n- Status: 'started'\n- chunks:[\n {\n id as fingerprint,\n status,\n s3Link\n }\n]\n- Size\n- OwnerId\n- S3Link\n- Creation Timestamp\n- Update Timestamp\n- ...metadata", "fontSize": 20, "fontFamily": 5, "textAlign": "left", "verticalAlign": "top", "containerId": null, "originalText": "File Metadata\n- FileID\n- File Name\n- MimeType\n- Compression Algo\n- Status: 'started'\n- chunks:[\n {\n id as fingerprint,\n status,\n s3Link\n }\n]\n- Size\n- OwnerId\n- S3Link\n- Creation Timestamp\n- Update Timestamp\n- ...metadata", "autoResize": true, "lineHeight": 1.25 }, { "id": "CXuAuwpv-RYkihc6dcK6E", "type": "rectangle", "x": 2987.647905474241, "y": 1406.5436233685248, "width": 211.6682683596582, "height": 93.06057562542173, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b1n", "roundness": { "type": 3 }, "seed": 2001414218, "version": 154, "versionNonce": 918777558, "isDeleted": false, "boundElements": [ { "type": "text", "id": "kFs0X5hhpd5IR2ykoaBqe" }, { "id": "kYcdFwMO2g18KBc5U3dzx", "type": "arrow" }, { "id": "CgHtNwwVbxlsrM9x84VfC", "type": "arrow" }, { "id": "uFS0Z2YNhMC49CG1nfo8b", "type": "arrow" } ], "updated": 1768628668750, "link": null, "locked": false }, { "id": "kFs0X5hhpd5IR2ykoaBqe", "type": "text", "x": 3033.8520958064137, "y": 1440.5739111812356, "width": 119.2598876953125, "height": 25, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b1o", "roundness": null, "seed": 695213078, "version": 41, "versionNonce": 1096492054, "isDeleted": false, "boundElements": [], "updated": 1768628668750, "link": null, "locked": false, "text": "Sync Service", "fontSize": 20, "fontFamily": 5, "textAlign": "center", "verticalAlign": "middle", "containerId": "CXuAuwpv-RYkihc6dcK6E", "originalText": "Sync Service", "autoResize": true, "lineHeight": 1.25 }, { "id": "kYcdFwMO2g18KBc5U3dzx", "type": "arrow", "x": 2851.8223080725693, "y": 1254.1520470202306, "width": 133.54682267300313, "height": 155.0709594365842, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b1p", "roundness": { "type": 2 }, "seed": 178925590, "version": 91, "versionNonce": 292360202, "isDeleted": false, "boundElements": [], "updated": 1768628690720, "link": null, "locked": false, "points": [ [ 0, 0 ], [ 133.54682267300313, 155.0709594365842 ] ], "lastCommittedPoint": null, "startBinding": { "elementId": "BR2X28VQ55dV9-1aqPm9N", "focus": -0.12603071705052504, "gap": 8.916903308674591 }, "endBinding": { "elementId": "CXuAuwpv-RYkihc6dcK6E", "focus": -0.4819153446221646, "gap": 8.4323180903707 }, "startArrowhead": null, "endArrowhead": "arrow", "elbowed": false }, { "id": "FKgf2ItAmvz3Z33UsY-TC", "type": "arrow", "x": 2857.393371973851, "y": 1073.7813596236128, "width": 271.1643130045236, "height": 80.74158125171812, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b1q", "roundness": { "type": 2 }, "seed": 854837910, "version": 45, "versionNonce": 378888854, "isDeleted": false, "boundElements": [ { "type": "text", "id": "gq2RQo5bESaf-gxCCPxGW" } ], "updated": 1767419829712, "link": null, "locked": false, "points": [ [ 0, 0 ], [ 271.1643130045236, -80.74158125171812 ] ], "lastCommittedPoint": null, "startBinding": { "elementId": "BR2X28VQ55dV9-1aqPm9N", "focus": -0.19067876068325684, "gap": 9.17316767900411 }, "endBinding": { "elementId": "uWSC4vhMFzkrAjqvSWrpH", "focus": 0.5338338722181384, "gap": 12.812865797680843 }, "startArrowhead": null, "endArrowhead": "arrow", "elbowed": false }, { "id": "gq2RQo5bESaf-gxCCPxGW", "type": "text", "x": 2924.8955876802147, "y": 995.9105689977537, "width": 136.15988159179688, "height": 75, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b1qV", "roundness": null, "seed": 1352603274, "version": 53, "versionNonce": 2119779402, "isDeleted": false, "boundElements": [], "updated": 1767419829021, "link": null, "locked": false, "text": "upload(file,\nfileMetadat)\ngetFile(fileId)", "fontSize": 20, "fontFamily": 5, "textAlign": "center", "verticalAlign": "middle", "containerId": "FKgf2ItAmvz3Z33UsY-TC", "originalText": "upload(file,\nfileMetadat)\ngetFile(fileId)", "autoResize": true, "lineHeight": 1.25 }, { "id": "323buhPH76iXg5AMYkwjo", "type": "arrow", "x": 3387.640963381644, "y": 1030.9730067133087, "width": 265.42887233524107, "height": 99.27193583960229, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b1r", "roundness": { "type": 2 }, "seed": 1206880906, "version": 50, "versionNonce": 304995914, "isDeleted": false, "boundElements": [ { "type": "text", "id": "rwI-cM1f7sjZbG3FfrsjE" } ], "updated": 1767420042569, "link": null, "locked": false, "points": [ [ 0, 0 ], [ 265.42887233524107, 99.27193583960229 ] ], "lastCommittedPoint": null, "startBinding": { "elementId": "uWSC4vhMFzkrAjqvSWrpH", "focus": -0.2150747435682616, "gap": 7.903702570269161 }, "endBinding": { "elementId": "jhKgwi4-noczMtOqda8nw", "focus": -0.40321994054887106, "gap": 10.94097823592671 }, "startArrowhead": null, "endArrowhead": "arrow", "elbowed": false }, { "id": "rwI-cM1f7sjZbG3FfrsjE", "type": "text", "x": 3445.375457227487, "y": 1068.10897463311, "width": 149.9598846435547, "height": 25, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b1rV", "roundness": null, "seed": 259229846, "version": 16, "versionNonce": 673398998, "isDeleted": false, "boundElements": [], "updated": 1767420038943, "link": null, "locked": false, "text": "write metadata", "fontSize": 20, "fontFamily": 5, "textAlign": "center", "verticalAlign": "middle", "containerId": "323buhPH76iXg5AMYkwjo", "originalText": "write metadata", "autoResize": true, "lineHeight": 1.25 }, { "id": "W74cwodMrOsSaCeCJuvXy", "type": "text", "x": 2014.2774920566985, "y": 960.6528431883296, "width": 358.19976806640625, "height": 300, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b1s", "roundness": null, "seed": 394143958, "version": 339, "versionNonce": 1967408074, "isDeleted": false, "boundElements": [], "updated": 1767419924118, "link": null, "locked": false, "text": "1. Remote Changed\n- pull for changes\n- download the new file and replace\n\n2. Local Changed\n- upload the changed files to remote\n\nWindows API\n- FileSystemWatcher\n\nMacOS API:\n- FSEvents", "fontSize": 20, "fontFamily": 5, "textAlign": "left", "verticalAlign": "top", "containerId": null, "originalText": "1. Remote Changed\n- pull for changes\n- download the new file and replace\n\n2. Local Changed\n- upload the changed files to remote\n\nWindows API\n- FileSystemWatcher\n\nMacOS API:\n- FSEvents", "autoResize": true, "lineHeight": 1.25 }, { "id": "Pb-gZrIBeE29nE1JZ68Ma", "type": "rectangle", "x": 2390.9943179501884, "y": 1028.7032365761033, "width": 150.39057618773904, "height": 342.05191906380446, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "dashed", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b1t", "roundness": { "type": 3 }, "seed": 1043450390, "version": 149, "versionNonce": 1528899018, "isDeleted": false, "boundElements": [ { "id": "6oZcvla33sgDc669OjQQt", "type": "arrow" }, { "id": "pw8kRWNP25SLmtcK8nAU2", "type": "arrow" } ], "updated": 1768628537808, "link": null, "locked": false }, { "id": "35X4xRWYiSQp2jbU_mC19", "type": "rectangle", "x": 2424.352861605508, "y": 1133.3888222304652, "width": 94.6957863694297, "height": 74.41429191762677, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b1u", "roundness": { "type": 3 }, "seed": 526054858, "version": 105, "versionNonce": 967661654, "isDeleted": false, "boundElements": [ { "type": "text", "id": "rT0ik9x_NhPHOMSi2UHLq" } ], "updated": 1768628429374, "link": null, "locked": false }, { "id": "rT0ik9x_NhPHOMSi2UHLq", "type": "text", "x": 2444.5607782887582, "y": 1145.5959681892784, "width": 54.27995300292969, "height": 50, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b1v", "roundness": null, "seed": 935267990, "version": 61, "versionNonce": 1917574550, "isDeleted": false, "boundElements": [], "updated": 1768628429374, "link": null, "locked": false, "text": "Client\nApp", "fontSize": 20, "fontFamily": 5, "textAlign": "center", "verticalAlign": "middle", "containerId": "35X4xRWYiSQp2jbU_mC19", "originalText": "Client App", "autoResize": true, "lineHeight": 1.25 }, { "id": "aaQHUaFQ6k6-2AGt0dFxD", "type": "rectangle", "x": 2417.0457622907475, "y": 1223.3909396351091, "width": 104.48874623559868, "height": 48.427864459675675, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b1w", "roundness": { "type": 3 }, "seed": 1129275350, "version": 61, "versionNonce": 1701906570, "isDeleted": false, "boundElements": [ { "type": "text", "id": "S-xTpTIx_u2GkhKNmanTJ" } ], "updated": 1767419746168, "link": null, "locked": false }, { "id": "S-xTpTIx_u2GkhKNmanTJ", "type": "text", "x": 2427.4601640950705, "y": 1235.104871864947, "width": 83.65994262695312, "height": 25, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b1x", "roundness": null, "seed": 1650231254, "version": 10, "versionNonce": 1373688918, "isDeleted": false, "boundElements": [], "updated": 1767419747762, "link": null, "locked": false, "text": "Local Db", "fontSize": 20, "fontFamily": 5, "textAlign": "center", "verticalAlign": "middle", "containerId": "aaQHUaFQ6k6-2AGt0dFxD", "originalText": "Local Db", "autoResize": true, "lineHeight": 1.25 }, { "id": "Lo6n4Tz0vb_4zBQ9Bnozi", "type": "rectangle", "x": 2414.1048235645835, "y": 1299.1841779264146, "width": 100.12615040736773, "height": 60, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b1y", "roundness": { "type": 3 }, "seed": 1038890710, "version": 46, "versionNonce": 1758302294, "isDeleted": false, "boundElements": [ { "type": "text", "id": "vL86iYYQGJKBYSKkeFbxa" } ], "updated": 1767419759834, "link": null, "locked": false }, { "id": "vL86iYYQGJKBYSKkeFbxa", "type": "text", "x": 2434.0679231823297, "y": 1304.1841779264146, "width": 60.199951171875, "height": 50, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b1z", "roundness": null, "seed": 1990686614, "version": 23, "versionNonce": 1124149270, "isDeleted": false, "boundElements": [], "updated": 1767419760686, "link": null, "locked": false, "text": "Local\nFolder", "fontSize": 20, "fontFamily": 5, "textAlign": "center", "verticalAlign": "middle", "containerId": "Lo6n4Tz0vb_4zBQ9Bnozi", "originalText": "Local Folder", "autoResize": true, "lineHeight": 1.25 }, { "id": "6oZcvla33sgDc669OjQQt", "type": "arrow", "x": 2545.4335265636705, "y": 1187.6320309152018, "width": 69.83982790723803, "height": 2.0775248157690385, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b20", "roundness": { "type": 2 }, "seed": 1582996118, "version": 114, "versionNonce": 474864778, "isDeleted": false, "boundElements": [], "updated": 1768628537808, "link": null, "locked": false, "points": [ [ 0, 0 ], [ 69.83982790723803, -2.0775248157690385 ] ], "lastCommittedPoint": null, "startBinding": { "elementId": "Pb-gZrIBeE29nE1JZ68Ma", "focus": -0.05615297338603289, "gap": 9.530593963211686 }, "endBinding": { "elementId": "BR2X28VQ55dV9-1aqPm9N", "focus": -0.05141042094750814, "gap": 13.646711844792208 }, "startArrowhead": "arrow", "endArrowhead": "arrow", "elbowed": false }, { "id": "s9KQ5lIG19avv8P3GAlde", "type": "text", "x": 2354.0291333201635, "y": 1399.4933743125885, "width": 344.0596923828125, "height": 175, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b21", "roundness": null, "seed": 1880101718, "version": 169, "versionNonce": 155847318, "isDeleted": false, "boundElements": [], "updated": 1767420020135, "link": null, "locked": false, "text": "Upload large files: 50gb\n\n50gb @ 100mbps = 1hr 12 min\n\nchunk files on client to 5mb chunk\n\nfingerprint each chunk: hash(bytes)", "fontSize": 20, "fontFamily": 5, "textAlign": "left", "verticalAlign": "top", "containerId": null, "originalText": "Upload large files: 50gb\n\n50gb @ 100mbps = 1hr 12 min\n\nchunk files on client to 5mb chunk\n\nfingerprint each chunk: hash(bytes)", "autoResize": true, "lineHeight": 1.25 }, { "id": "pw8kRWNP25SLmtcK8nAU2", "type": "arrow", "x": 2469.006895839052, "y": 1018.6680070068968, "width": 1161.1577865467343, "height": 367.41168026322237, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b22", "roundness": { "type": 2 }, "seed": 913038346, "version": 229, "versionNonce": 1815460682, "isDeleted": false, "boundElements": [ { "type": "text", "id": "lSXa63iYEBTNgkCpaxvPd" } ], "updated": 1768628537808, "link": null, "locked": false, "points": [ [ 0, 0 ], [ 281.933036393345, -367.41168026322237 ], [ 1161.1577865467343, -215.20284732665516 ] ], "lastCommittedPoint": null, "startBinding": { "elementId": "Pb-gZrIBeE29nE1JZ68Ma", "focus": -0.6593934879619889, "gap": 11.104789380950479 }, "endBinding": { "elementId": "ZKHtfjRND0MAQt1UjvFzi", "focus": -0.16307302380542354, "gap": 10.910970376903236 }, "startArrowhead": "arrow", "endArrowhead": "arrow", "elbowed": false }, { "id": "lSXa63iYEBTNgkCpaxvPd", "type": "text", "x": 2530.810064678686, "y": 626.2563267436744, "width": 440.2597351074219, "height": 50, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b23", "roundness": null, "seed": 874466262, "version": 74, "versionNonce": 1211426902, "isDeleted": false, "boundElements": [], "updated": 1767419917865, "link": null, "locked": false, "text": "Download Directly from S3\nUpload directly to S3 with that presigend url", "fontSize": 20, "fontFamily": 5, "textAlign": "center", "verticalAlign": "middle", "containerId": "pw8kRWNP25SLmtcK8nAU2", "originalText": "Download Directly from S3\nUpload directly to S3 with that presigend url", "autoResize": true, "lineHeight": 1.25 }, { "id": "ZUo4DIbYaf26sFTRSgHv0", "type": "text", "x": 3801.1296264411585, "y": 781.7131958393427, "width": 334.9597473144531, "height": 75, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b24", "roundness": null, "seed": 1238874582, "version": 63, "versionNonce": 1951499338, "isDeleted": false, "boundElements": [], "updated": 1767420132320, "link": null, "locked": false, "text": "Storing the raw bytes of the files\n\ns3: multi-part upload", "fontSize": 20, "fontFamily": 5, "textAlign": "left", "verticalAlign": "top", "containerId": null, "originalText": "Storing the raw bytes of the files\n\ns3: multi-part upload", "autoResize": true, "lineHeight": 1.25 }, { "id": "CgHtNwwVbxlsrM9x84VfC", "type": "arrow", "x": 3233.635320334006, "y": 1071.7020571133974, "width": 137.03894829585124, "height": 334.1778621860394, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b25", "roundness": { "type": 2 }, "seed": 1405548618, "version": 81, "versionNonce": 1399166614, "isDeleted": false, "boundElements": [], "updated": 1768628668750, "link": null, "locked": false, "points": [ [ 0, 0 ], [ -137.03894829585124, 334.1778621860394 ] ], "lastCommittedPoint": null, "startBinding": { "elementId": "uWSC4vhMFzkrAjqvSWrpH", "focus": 0.001121289921484874, "gap": 2.324361747507055 }, "endBinding": { "elementId": "CXuAuwpv-RYkihc6dcK6E", "focus": -0.1299329999536972, "gap": 1.6230076787544476 }, "startArrowhead": "arrow", "endArrowhead": "arrow", "elbowed": false }, { "id": "Rc8YBN087bJ_bRhrIGn2B", "type": "arrow", "x": 3391.6679749154027, "y": 993.7902668850033, "width": 301.4340163665556, "height": 122.11607399470176, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b26", "roundness": { "type": 2 }, "seed": 1035030602, "version": 92, "versionNonce": 1533114390, "isDeleted": false, "boundElements": [ { "type": "text", "id": "Tar4C0EkeROV0PISZrG-M" } ], "updated": 1768622832296, "link": null, "locked": false, "points": [ [ 0, 0 ], [ 171.1845993802017, -18.280191750181075 ], [ 301.4340163665556, -122.11607399470176 ] ], "lastCommittedPoint": null, "startBinding": { "elementId": "uWSC4vhMFzkrAjqvSWrpH", "focus": 0.013504164369049648, "gap": 11.930714104028084 }, "endBinding": { "elementId": "ZKHtfjRND0MAQt1UjvFzi", "focus": -0.5827846836051463, "gap": 7.957992263856378 }, "startArrowhead": "arrow", "endArrowhead": "arrow", "elbowed": false }, { "id": "Tar4C0EkeROV0PISZrG-M", "type": "text", "x": 3490.672635025585, "y": 963.0100751348223, "width": 144.35987854003906, "height": 25, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b26V", "roundness": null, "seed": 1451353750, "version": 19, "versionNonce": 134496982, "isDeleted": false, "boundElements": [], "updated": 1768622830775, "link": null, "locked": false, "text": "s3 notification", "fontSize": 20, "fontFamily": 5, "textAlign": "center", "verticalAlign": "middle", "containerId": "Rc8YBN087bJ_bRhrIGn2B", "originalText": "s3 notification", "autoResize": true, "lineHeight": 1.25 }, { "id": "logk2tM1TasHn6V0gBvsY", "type": "text", "x": 2612.2215879599116, "y": 1729.4886649056468, "width": 414.01971435546875, "height": 225, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b28", "roundness": null, "seed": 680362378, "version": 175, "versionNonce": 209919498, "isDeleted": false, "boundElements": [], "updated": 1767421342785, "link": null, "locked": false, "text": "Sync\n1. Sync to be fast\n- adaptive polling\n- delta sync to only detch changed chunks\n\n2. Sync to be consistent\n- polling the DB or event bus w/ cursor\n- reconciliation\n ", "fontSize": 20, "fontFamily": 5, "textAlign": "left", "verticalAlign": "top", "containerId": null, "originalText": "Sync\n1. Sync to be fast\n- adaptive polling\n- delta sync to only detch changed chunks\n\n2. Sync to be consistent\n- polling the DB or event bus w/ cursor\n- reconciliation\n ", "autoResize": true, "lineHeight": 1.25 }, { "id": "gqwnCnCvOqUwe7shyLlws", "type": "rectangle", "x": 3308.4972801922136, "y": 1431.6362481910794, "width": 230.41827812211022, "height": 73.86515398120605, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b29", "roundness": { "type": 3 }, "seed": 707935382, "version": 80, "versionNonce": 1568045590, "isDeleted": false, "boundElements": [ { "type": "text", "id": "ynF5qREGpM5mD_MhAl-Dd" }, { "id": "uFS0Z2YNhMC49CG1nfo8b", "type": "arrow" }, { "id": "3b757-G7rHCbnqi5oTG3-", "type": "arrow" } ], "updated": 1768628666541, "link": null, "locked": false }, { "id": "ynF5qREGpM5mD_MhAl-Dd", "type": "text", "x": 3372.7064497708466, "y": 1456.0688251816823, "width": 101.99993896484375, "height": 25, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b2A", "roundness": null, "seed": 1697997910, "version": 22, "versionNonce": 202032982, "isDeleted": false, "boundElements": [], "updated": 1768628666541, "link": null, "locked": false, "text": "Event Bus", "fontSize": 20, "fontFamily": 5, "textAlign": "center", "verticalAlign": "middle", "containerId": "gqwnCnCvOqUwe7shyLlws", "originalText": "Event Bus", "autoResize": true, "lineHeight": 1.25 }, { "id": "uFS0Z2YNhMC49CG1nfo8b", "type": "arrow", "x": 3205.0670116415095, "y": 1461.3756738632387, "width": 102.3342902869208, "height": 4.434710249056479, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b2B", "roundness": { "type": 2 }, "seed": 1479899286, "version": 82, "versionNonce": 518149078, "isDeleted": false, "boundElements": [], "updated": 1768628668750, "link": null, "locked": false, "points": [ [ 0, 0 ], [ 102.3342902869208, 4.434710249056479 ] ], "lastCommittedPoint": null, "startBinding": { "elementId": "CXuAuwpv-RYkihc6dcK6E", "focus": 0.06970692107670068, "gap": 6.357796997226842 }, "endBinding": { "elementId": "gqwnCnCvOqUwe7shyLlws", "focus": -0.0562583832031241, "gap": 1.153189666483513 }, "startArrowhead": "arrow", "endArrowhead": "arrow", "elbowed": false }, { "id": "3b757-G7rHCbnqi5oTG3-", "type": "arrow", "x": 3383.2678967542774, "y": 1067.5721217201908, "width": 136.0940049828746, "height": 362.78877909334574, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b2C", "roundness": { "type": 2 }, "seed": 1863313686, "version": 82, "versionNonce": 1450535382, "isDeleted": false, "boundElements": [], "updated": 1768628666541, "link": null, "locked": false, "points": [ [ 0, 0 ], [ 98.74547176757596, 183.672314356701 ], [ 136.0940049828746, 362.78877909334574 ] ], "lastCommittedPoint": null, "startBinding": { "elementId": "uWSC4vhMFzkrAjqvSWrpH", "focus": -0.5864390023498247, "gap": 12.106685135696871 }, "endBinding": { "elementId": "gqwnCnCvOqUwe7shyLlws", "focus": 0.843074576772088, "gap": 3.9309847656337102 }, "startArrowhead": "arrow", "endArrowhead": "arrow", "elbowed": false }, { "id": "Z7gccOtcLtw6HXMRUAdSR", "type": "text", "x": 4254.522312880563, "y": 997.4999987221589, "width": 214.39981079101562, "height": 50, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b2D", "roundness": null, "seed": 585585302, "version": 35, "versionNonce": 1623668554, "isDeleted": false, "boundElements": [], "updated": 1767421220734, "link": null, "locked": false, "text": "Folder\n- Cursor (sync cursor)", "fontSize": 20, "fontFamily": 5, "textAlign": "left", "verticalAlign": "top", "containerId": null, "originalText": "Folder\n- Cursor (sync cursor)", "autoResize": true, "lineHeight": 1.25 }, { "id": "MxmKk3Fu-W6kUuJn664-v", "type": "text", "x": 3311.0562249983786, "y": 1539.2199485188253, "width": 308.259765625, "height": 25, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b2E", "roundness": null, "seed": 1756447626, "version": 84, "versionNonce": 425411094, "isDeleted": false, "boundElements": [], "updated": 1768628670174, "link": null, "locked": false, "text": "- partioning on userId, folderId", "fontSize": 20, "fontFamily": 5, "textAlign": "left", "verticalAlign": "top", "containerId": null, "originalText": "- partioning on userId, folderId", "autoResize": true, "lineHeight": 1.25 }, { "id": "5FdTWRSCaD_QgBoltpj7k", "type": "text", "x": 4291.430658340313, "y": 1123.9482830678016, "width": 154.21987915039062, "height": 375, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b2F", "roundness": null, "seed": 554779722, "version": 190, "versionNonce": 1488424970, "isDeleted": false, "boundElements": null, "updated": 1768628505648, "link": null, "locked": false, "text": "Permissions\n- permission_id\n- resource_type\n- resource_id\n- userId\n- role\n- created_at\n... metadata\n\nChunks\n- chunk_id\n- object_key\n- size_bytes\n- checksum\n- created_at", "fontSize": 20, "fontFamily": 5, "textAlign": "left", "verticalAlign": "top", "containerId": null, "originalText": "Permissions\n- permission_id\n- resource_type\n- resource_id\n- userId\n- role\n- created_at\n... metadata\n\nChunks\n- chunk_id\n- object_key\n- size_bytes\n- checksum\n- created_at", "autoResize": true, "lineHeight": 1.25 }, { "id": "134ohjFzLHSXs5TPP7RgV", "type": "rectangle", "x": 2426.6150196105427, "y": 1054.6718400322518, "width": 88.0961410913169, "height": 59.67751196221525, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b2G", "roundness": { "type": 3 }, "seed": 1849880778, "version": 31, "versionNonce": 673800726, "isDeleted": false, "boundElements": [ { "type": "text", "id": "Z7QlJ4m68CLFCPhdfcb6Z" } ], "updated": 1768628544585, "link": null, "locked": false }, { "id": "Z7QlJ4m68CLFCPhdfcb6Z", "type": "text", "x": 2432.543125556592, "y": 1072.0105960133594, "width": 76.23992919921875, "height": 25, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b2H", "roundness": null, "seed": 850981770, "version": 9, "versionNonce": 498567242, "isDeleted": false, "boundElements": null, "updated": 1768628546549, "link": null, "locked": false, "text": "watcher", "fontSize": 20, "fontFamily": 5, "textAlign": "center", "verticalAlign": "middle", "containerId": "134ohjFzLHSXs5TPP7RgV", "originalText": "watcher", "autoResize": true, "lineHeight": 1.25 }, { "id": "R9QbGsIYy9-hSWsCqG9YZ", "type": "text", "x": 3848.5763697599828, "y": 1527.6666112619646, "width": 192.29983520507812, "height": 100, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b2I", "roundness": null, "seed": 1780555350, "version": 71, "versionNonce": 1571960278, "isDeleted": false, "boundElements": null, "updated": 1768628624908, "link": null, "locked": false, "text": "File_version_chunks\n- version_id\n- chunk_index\n- chunk_id", "fontSize": 20, "fontFamily": 5, "textAlign": "left", "verticalAlign": "top", "containerId": null, "originalText": "File_version_chunks\n- version_id\n- chunk_index\n- chunk_id", "autoResize": true, "lineHeight": 1.25 }, { "id": "SCI12MTNLXogHTsY0GdN1", "type": "rectangle", "x": 2983.0633642624093, "y": 1125.4448473424363, "width": 119.60445130353673, "height": 85, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b2J", "roundness": { "type": 3 }, "seed": 1365334346, "version": 57, "versionNonce": 821119946, "isDeleted": false, "boundElements": [ { "id": "j9PmhfB6juRFVJH4csWhw", "type": "arrow" }, { "id": "DgIi9RZCqHsba14sUi6kV", "type": "arrow" }, { "type": "text", "id": "AAUx6TPBQ0T55lLIspo-0" } ], "updated": 1768628654054, "link": null, "locked": false }, { "id": "AAUx6TPBQ0T55lLIspo-0", "type": "text", "x": 2996.815617379998, "y": 1130.4448473424363, "width": 92.09994506835938, "height": 75, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b2JV", "roundness": null, "seed": 1612226378, "version": 23, "versionNonce": 3092950, "isDeleted": false, "boundElements": null, "updated": 1768628680463, "link": null, "locked": false, "text": "Read/\nDownload\nSVC", "fontSize": 20, "fontFamily": 5, "textAlign": "center", "verticalAlign": "middle", "containerId": "SCI12MTNLXogHTsY0GdN1", "originalText": "Read/ Download SVC", "autoResize": true, "lineHeight": 1.25 }, { "id": "j9PmhfB6juRFVJH4csWhw", "type": "arrow", "x": 2853.5220480170724, "y": 1131.3184598181565, "width": 124.46426217385806, "height": 29.231279622317288, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b2K", "roundness": { "type": 2 }, "seed": 624142218, "version": 37, "versionNonce": 292241034, "isDeleted": false, "boundElements": null, "updated": 1768628643302, "link": null, "locked": false, "points": [ [ 0, 0 ], [ 124.46426217385806, 29.231279622317288 ] ], "lastCommittedPoint": null, "startBinding": { "elementId": "rX7h2-Y5RvGJdgrtOgVrh", "focus": -0.6301911177030953, "gap": 31.506474622852693 }, "endBinding": { "elementId": "SCI12MTNLXogHTsY0GdN1", "focus": -0.4799764630254718, "gap": 5.0770540714788694 }, "startArrowhead": null, "endArrowhead": "arrow", "elbowed": false }, { "id": "DgIi9RZCqHsba14sUi6kV", "type": "arrow", "x": 3109.5793677805264, "y": 1149.4059678255799, "width": 77.63628325784202, "height": 62.912021846104835, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b2L", "roundness": { "type": 2 }, "seed": 452131222, "version": 35, "versionNonce": 267365718, "isDeleted": false, "boundElements": null, "updated": 1768628647399, "link": null, "locked": false, "points": [ [ 0, 0 ], [ 77.63628325784202, -62.912021846104835 ] ], "lastCommittedPoint": null, "startBinding": { "elementId": "SCI12MTNLXogHTsY0GdN1", "focus": 0.6232162158318286, "gap": 6.911552214580297 }, "endBinding": { "elementId": "uWSC4vhMFzkrAjqvSWrpH", "focus": -0.12659378878173985, "gap": 15.903365736388423 }, "startArrowhead": null, "endArrowhead": "arrow", "elbowed": false }, { "id": "rrbrspedNQIVJSNwjfOk9", "type": "text", "x": 2990.1277913222484, "y": 1226.8169618377783, "width": 108.07991027832031, "height": 50, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "index": "b2M", "roundness": null, "seed": 1869927498, "version": 47, "versionNonce": 2088737814, "isDeleted": false, "boundElements": null, "updated": 1768628704892, "link": null, "locked": false, "text": "- generate \nsigned-url", "fontSize": 20, "fontFamily": 5, "textAlign": "left", "verticalAlign": "top", "containerId": null, "originalText": "- generate \nsigned-url", "autoResize": true, "lineHeight": 1.25 } ], "appState": { "gridSize": 20, "gridStep": 5, "gridModeEnabled": false, "viewBackgroundColor": "#ffffff" }, "files": {} }