Application for Filecoin ProPGF Batch 2

ProbeLab Gauge for FOC and Retrieval Testing

This project has the following two goals: - Develop tooling to measure the retrieval success rate from the Filecoin network - Develop tooling and produce metrics to assess the performance of Filecoin Onchain Cloud. The outcome of this project will come in the form of live dashboards, hosted on https://probelab.io.

https://probelab.io/networks/

Yiannis Psarras

yiannisbot

yiannis (ProbeLab)

Having accurate metrics for the performance of FOC is the most important aspect of FOC as a product. Metrics will allow the FOC developer team to provide accurate Service Level Agreements (SLAs) to their customers. Without metrics to showcase the performance of the system: i) the development and maintenance team does not have visibility on what needs attention/improvement, and ii) users do not know what to expect in terms of performance, when using or building on this system. In this project we will provide all the necessary tools and dashboards to guide product success and adoption, but also drive the right improvement and optimisation efforts.

The outcome of this project will benefit: i) FOC (and perhaps more generally Filecoin) engineers, developers and maintainers, ii) Storage Providers, who will be able to identify performance challenges and improve their deployments, and iii) end-users and builders who are using (or plan to use) FOC by having guarantees on SLAs.

1. Tooling to measure the retrieval performance of the Filecoin network. The tooling will either be built from scratch, or will be developed on top of existing tooling (e.g., Spark Retrieval Tester, Dealbot). The tool will work on the PieceCID level to guarantee accurate performance measurement and will replace the existing dealbot: https://dealbot.fwss.io/ 2. Dashboards on https://probelab.io for the following metrics: retrievability success rate, node RTT, fetch latency (TTFB and TTLB), throughput, node uptime, connection errors and related metrics, ideally measured from several different geographic locations.

$100.000 USD

N/A

$150,000 USD

The alternative metric here is the number of FOC users and, therefore, indirectly the number of clients that have more active data stored with Filecoin.

- A stable and accurate retrieval tester will improve the credibility of Filecoin as a distributed storage system. High retrieval success rate will drive adoption. Low retrieval success rate will force developers and Storage Providers to optimise their deployment. - Accurate metrics for the Filecoin Onchain Cloud related to the retrievability of content as well as the performance of retrievals will inform developers and maintainers on where attention is needed in order to improve performance, but most importantly will provide the right SLA guarantees to FOC users. In turn, this will drive adoption and establish FOC as a robust product in the decentralised storage space.

- Visibility of FOC-specific metrics and dashboards on our website: 200 unique visitors per month tracked through https://plausible.io. - No downtime for tooling and monitoring of retrievals and FOC metrics, so that our dashboards are always up-to-date.

The majority of the funding requested in this round is for the development of tooling and data analysis software and the set up of the related monitoring infrastructure. Unless costs are exceedingly higher than our current projection, the maintenance and infrastructure cost for 2026 will be covered by ProbeLab. We will apply for follow-up maintenance and infrastructure cost for 2027 through ProPGF. The yearly maintenance and infrastructure cost is expected to be in the area of 10%-20% of the current grant request.

Dennis Trautwein | Software and Infrastructure Engineer | Dennis has led the development of key ProbeLab tooling and has set up the majority of our monitoring infrastructure that runs 24/7. He has also built the dashboards found at: [probelab.io](http://probelab.io). Mikel Cortes | Softare and Data Analysis Enginner | Mikel has built several of ProbeLab’s core tools and the majority of our data analysis scripts and software. Yiannis Psarras | Team Lead | Yiannis has founded and led the ProbeLab team since 2022 to become the flagship team for metrics and measurements in the wider Web3 space. ProbeLab Team: Our team, ProbeLab, specialises in developing tools for measurement and monitoring of P2P layer protocols. We've got a wide variety of tools and deployed infrastructure that runs 24/7 and monitors several high profile networks - see: https://probelab.io/ for details. As a sample of our track-record to date, we monitor more than 10 networks, performing more than 2k crawls per week, monitoring more than 50k nodes, for the past 3 years that we’ve been in operation. You can find our results and dashboards at: https://probelab.io, which has become the reference website for metrics in the wider Web3 space, given the number of high-profile networks we're monitoring.

We will attempt to re-use existing retrieval testing software (e.g., Dealbot, Spark) and improve its functionality in order to meet the requirements, but at the same time save time (and money) by avoiding to build everything from scratch. However, if we figure out that repurposing and re-using code is not possible, or beneficial in terms of effort needed, we will have to course-adjust and consider a full-fledged retrieval tester implementation. We anticipate that the costing of this project won’t change significantly in either case. The timing, however, might be slightly affected and have to be extended to accommodate the change of plans.

Website: https://probelab.io GH Org: https://github.com/probe-lab/ Core Tools: - Hermes (Gossipsub Listener and Tracer): https://github.com/probe-lab/hermes - Bitswap Sniffer: https://github.com/probe-lab/bitswap-sniffer - Ants (DHT Client monitor): https://github.com/probe-lab/ants-watch - Ethereum DAS Guardian: https://github.com/probe-lab/eth-das-guardian - Tiros (IPFS Website measurement tool): https://github.com/probe-lab/tiros - Parsec (DHT and IPNI performance measurement tool): https://github.com/probe-lab/parsec - Nebula Crawler (DHT and Gossipsub crawler supporting multiple networks): https://github.com/dennis-tra/nebula ethresear.ch posts on P2P-related matters - Gossip IWANT/IHAVE Effectiveness in Ethereum’s Gossipsusb network: https://ethresear.ch/t/gossip-iwant-ihave-effectiveness-in-ethereums-gossipsusb-network/19686 - Number Duplicate Messages in Ethereum’s Gossipsub Network: https://ethresear.ch/t/number-duplicate-messages-in-ethereums-gossipsub-network/19921/ - Gossipsub Network Dynamicity through GRAFTs and PRUNEs: https://ethresear.ch/t/gossipsub-network-dynamicity-through-grafts-and-prunes/19750 - Ethereum Node Message Propagation Bandwidth Consumption: https://ethresear.ch/t/ethereum-node-message-propagation-bandwidth-consumption/19952/ - Gossipsub Message Propagation Latency: https://ethresear.ch/t/gossipsub-message-propagation-latency/19982

FilOz team | Will monitor metrics and use results to inform protocol and FOC design decisions. Filecoin Foundation | Will monitor metrics and use results to identify poor-performing Storage Providers