My name is
Krzysztof Lis
and
I am a PhD student at the EPFL
Computer Vision Lab
of Pascal Fua.
I come from Poland and did my master's degree at ETH Zurich.
My research concerns visual perception for autonomous cars:
detecting unusual and anomalous objects,
semantic segmentation,
jointly perceiving semantic and 3D information about the scene.
Apart from computer vision, my interests are game development, digital art, fantasy & science-fiction.
SegmentMeIfYouCan: A Benchmark for Anomaly Segmentation
Robin Chan,
Krzysztof Lis
,
Svenja Uhlemeyer,
Hermann Blum,
Sina Honari,
Roland Siegwart,
Pascal Fua,
Mathieu Salzmann,
Matthias Rottmann
NeurIPS 2021 Datasets and Benchmarks
2021
State-of-the-art semantic or instance segmentation deep neural networks (DNNs) are usually trained on a closed set of semantic classes. As such, they are ill-equipped to handle previously-unseen objects.
However, detecting and localizing such objects is crucial for safety-critical applications such as perception for automated driving, especially if they appear on the road ahead. While some methods have tackled the tasks of anomalous or out-of-distribution object segmentation, progress remains slow, in large part due to the lack of solid benchmarks; existing datasets either consist of synthetic data, or suffer from label inconsistencies. In this paper, we bridge this gap by introducing the "SegmentMeIfYouCan" benchmark. Our benchmark addresses two tasks: Anomalous object segmentation, which considers any previously-unseen object category; and road obstacle segmentation, which focuses on any object on the road, may it be known or unknown.
We provide two corresponding datasets together with a test suite performing an in-depth method analysis, considering both established pixel-wise performance metrics and recent component-wise ones, which are insensitive to object sizes. We empirically evaluate multiple state-of-the-art baseline methods, including several models specifically designed for anomaly / obstacle segmentation, on our datasets and on public ones, using our test suite.
The anomaly and obstacle segmentation results show that our datasets contribute to the diversity and difficulty of both data landscapes.
@InProceedings{
SegmentMeIfYouCan,
title = \{\{SegmentMeIfYouCan: A Benchmark for Anomaly Segmentation\}\},
author = {Robin Chan and Krzysztof Lis and Svenja Uhlemeyer and Hermann Blum and Sina Honari and Roland Siegwart and Pascal Fua and Mathieu Salzmann and Matthias Rottmann},
booktitle= {Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track},
year = {2021},
url = {https://openreview.net/forum?id=OFiGmksrSz1}
}
Detecting Road Obstacles by Erasing Them
Krzysztof Lis
,
Sina Honari,
Pascal Fua,
Mathieu Salzmann
ArXiv
2020
Vehicles can encounter a myriad of obstacles on the road, and it is impossible to record them all beforehand to train a detector. Instead, we select image patches and inpaint them with the surrounding road texture, which tends to remove obstacles from those patches. We then uses a network trained to recognize discrepancies between the original patch and the inpainted one, which signals an erased obstacle.
We also contribute a new dataset for monocular road obstacle detection, and show that our approach outperforms the state-of-the-art methods on both our new dataset and the standard Fishyscapes Lost \& Found benchmark.
Detecting the Unexpected via Image Resynthesis
Krzysztof Lis
,
Krishna Nakka,
Pascal Fua,
Mathieu Salzmann
ICCV
2019
Classical semantic segmentation methods, including the recent deep learning ones, assume that all classes observed at test time have been seen during training.
In this paper, we tackle the more realistic scenario where unexpected objects of unknown classes can appear at test time.
The main trends in this area either leverage the notion of prediction uncertainty to flag the regions with low confidence as unknown, or rely on autoencoders and highlight poorly-decoded regions.
Having observed that, in both cases, the detected regions typically do not correspond to unexpected objects, in this paper, we introduce a drastically different strategy:
It relies on the intuition that the network will produce spurious labels in regions depicting unexpected objects.
Therefore, resynthesizing the image from the resulting semantic map will yield significant appearance differences with respect to the input image.
In other words, we translate the problem of detecting unknown classes to one of identifying poorly-resynthesized image regions.
We show that this outperforms both uncertainty- and autoencoder-based methods.
@InProceedings{Lis_2019_ICCV,
author = {Lis, Krzysztof and Nakka, Krishna and Fua, Pascal and Salzmann, Mathieu},
title = {Detecting the Unexpected via Image Resynthesis},
booktitle = {IEEE International Conference on Computer Vision (ICCV)},
month = {October},
year = {2019},
url = {https://infoscience.epfl.ch/record/269093?ln=en}
}
Geometric and Physical Constraints for Drone-Based Head Plane Crowd Density Estimation
Weizhe Liu,
Krzysztof Lis
,
Mathieu Salzmann,
Pascal Fua
IROS
2019
State-of-the-art methods for counting people in crowded scenes rely on deep networks to estimate crowd density in the image plane.
While useful for this purpose, this image- plane density has no immediate physical meaning because it is subject to perspective distortion.
This is a concern in sequences acquired by drones because the viewpoint changes often.
This distortion is usually handled implicitly by either learning scale- invariant features or estimating density in patches of different sizes, neither of which accounts for the fact that scale changes must be consistent over the whole scene.
In this paper, we explicitly model the scale changes and reason in terms of people per square-meter.
We show that feeding the perspective model to the network allows us to enforce global scale consistency and that this model can be obtained on the fly from the drone sensors.
In addition, it also enables us to enforce physically-inspired temporal consistency constraints that do not have to be learned.
This yields an algorithm that outperforms state-of-the-art methods in inferring crowd density from a moving drone camera especially when perspective effects are strong.
@article{head_plane_crowd_density_2018,
title={Geometric and Physical Constraints for Drone-Based Head Plane Crowd Density Estimation},
author = {Weizhe Liu and Krzysztof Lis and Mathieu Salzmann and Pascal Fua},
journal = {IROS},
year={2019}
url = {https://infoscience.epfl.ch/record/268343}
}
The algorithm attempts to find the foreground object in a user-selected bounding box.
It presents a suggested per-pixel mask, the user can then correct its mistakes and next suggestion
will take those into account.
Augmented Unreality
Augmented Unreality is a plugin for
Unreal Engine 4
which enables creation of augmented reality applications
by displaying a video stream from a camera inside the game
and tracking camera position using the
ArUco fiducial markers
.
I created it as a semester project at ETH Zürich.
Comparing Python, Go, and C++ on the N-Queens Problem
Using the XML-native database BaseX to process big XML data.
LaTeX cleaner
A tool for preparing LaTeX sources for submission to ArXiv.
Kubernetes Job Dashboard
Web dashboard for jobs running on EPFL’s Kubernetes cluster for my and others labs. Lists pods per user, the number of GPUs allocated, GPU memory and compute utilization. The backend is using an async Python HTTP server aiohttp and interfacing with the Kubernetes API. The frontend web-application uses Preact.
Conquering a 5 GiB XML file with XML-native databases
Using the XML-native database BaseX to process big XML data.