Academic

The Annotated Transformer

Thu, 09 May 2024 00:00:00 +0000

from IPython.core.display import Image
Image('https://www.python.org/static/community_logos/python-logo-master-v3-TM-flattened.png')

print("Welcome to Academic!")

Welcome to Academic!

Organize your notebooks

Place the notebooks that you would like to publish in a notebooks folder at the root of your website.

Import the notebooks into your site

pipx install academic
academic import 'notebooks/**.ipynb' content/post/ --verbose

The notebooks will be published to the folder you specify above. In this case, they will be published to your content/post/ folder.

PEEB, Part-based Bird Classifier with an Explainable and Editable Language Bottleneck

Fri, 01 Mar 2024 00:00:00 +0000

PEEB: Part-based Bird Classifier with an Explainable and Editable Language Bottleneck

Abstract

CLIP-based classifiers rely on the prompt containing a {class name} that is known to the text encoder. That is, CLIP performs poorly on new classes or the classes whose names rarely appear on the Internet (e.g., scientific names of birds). For fine-grained classification, we propose PEEB - an explainable and editable classifier to (1) express the class name into a set of pre-defined text descriptors that describe the visual parts of that class; and (2) match the embeddings of the detected parts to their textual descriptors in each class to compute a logit score for classification. In a zero-shot setting where the class names are unknown, PEEB outperforms CLIP by a large margin (~10x in accuracy). Compared to part-based classifiers, PEEB is not only the state-of-the-art on the supervised-learning setting (88.80% accuracy) but also the first to enable users to edit the class definitions to form a new classifier without retraining. Compared to concept bottleneck models, PEEB is also the state-of-the-art in both zero-shot and supervised learning settings.

PEEB: Part-based Image Classifiers with an Explainable and Editable Language Bottleneck

Mon, 01 Jan 2024 00:00:00 +0000

Blog with Jupyter Notebooks!

Sat, 04 Nov 2023 00:00:00 +0000

from IPython.core.display import Image
Image('https://www.python.org/static/community_logos/python-logo-master-v3-TM-flattened.png')

print("Welcome to Academic!")

Welcome to Academic!

Organize your notebooks

Place the notebooks that you would like to publish in a notebooks folder at the root of your website.

Import the notebooks into your site

pipx install academic
academic import 'notebooks/**.ipynb' content/post/ --verbose

The notebooks will be published to the folder you specify above. In this case, they will be published to your content/post/ folder.

gScoreCAM, What objects is CLIP looking at?

Mon, 01 Aug 2022 00:00:00 +0000

gScoreCAM: What is CLIP looking at?

Abstract

Large-scale, multimodal models trained on web data such as OpenAI’s CLIP are becoming the foundation of many applications. Yet, they are also more complex to understand, test, and therefore align with human values. In this paper, we propose gScoreCAM–a state-of-the-art method for visualizing the main objects that CLIP is looking at in an image. On zero-shot object detection, gScoreCAM performs similarly to ScoreCAM, the best prior art on CLIP, yet 8 to 10 times faster. Our method outperforms other existing, well-known methods (HilaCAM, RISE, and the entire CAM family) by a large margin, especially in multi-object scenes. gScoreCAM sub-samples k = 300 channels (from 3,072 channels–i.e. reducing complexity by almost 10 times) of the highest gradients and linearly combines them into a final “attention” visualization. We demonstrate the utility and superiority of our method on three datasets: ImageNet, COCO, and PartImageNet. Our work opens up interesting future directions in understanding and de-biasing CLIP.

How explainable are adversarially-robust cnns?

Mon, 01 Aug 2022 00:00:00 +0000

How explainable are adversarially-robust cnns?

Abstract

Three important criteria of existing convolutional neural networks (CNNs) are (1) test-set accuracy; (2) out-of-distribution accuracy; and (3) explainability. While these criteria have been studied independently, their relationship is unknown. For example, do CNNs that have a stronger out-of-distribution performance have also stronger explainability? Furthermore, most prior feature-importance studies only evaluate methods on 2-3 common vanilla ImageNet-trained CNNs, leaving it unknown how these methods generalize to CNNs of other architectures and training algorithms. Here, we perform the first, large-scale evaluation of the relations of the three criteria using 9 feature-importance methods and 12 ImageNet-trained CNNs that are of 3 training algorithms and 5 CNN architectures. We find several important insights and recommendations for ML practitioners. First, adversarially robust CNNs have a higher explainability score on gradient-based attribution methods (but not CAM-based or perturbation-based methods). Second, AdvProp models, despite being highly accurate more than both vanilla and robust models alone, are not superior in explainability. Third, among 9 feature attribution methods tested, GradCAM and RISE are consistently the best methods. Fourth, Insertion and Deletion are biased towards vanilla and robust models respectively, due to their strong correlation with the confidence score distributions of a CNN. Fifth, we did not find a single CNN to be the best in all three criteria, which interestingly suggests that CNNs are harder to interpret as they become more accurate.

gScoreCAM: What objects is CLIP looking at?

Sat, 01 Jan 2022 00:00:00 +0000

How explainable are adversarially-robust cnns?

Sat, 01 Jan 2022 00:00:00 +0000

Welcome to Hugo Blox Builder, the website builder for Hugo

Sun, 13 Dec 2020 00:00:00 +0000

import libr
print('hello')

Overview

The Wowchemy website builder for Hugo, along with its starter templates, is designed for professional creators, educators, and teams/organizations - although it can be used to create any kind of site
The template can be modified and customised to suit your needs. It’s a good platform for anyone looking to take control of their data and online identity whilst having the convenience to start off with a no-code solution (write in Markdown and customize with YAML parameters) and having flexibility to later add even deeper personalization with HTML and CSS
You can work with all your favourite tools and apps with hundreds of plugins and integrations to speed up your workflows, interact with your readers, and much more

Get Started

👉 Create a new site
📚 Personalize your site
💬 Chat with the Wowchemy community or Hugo community
🐦 Twitter: @wowchemy @GeorgeCushen #MadeWithWowchemy
💡 Request a feature or report a bug for Wowchemy
⬆️ Updating Wowchemy? View the Update Tutorial and Release Notes

Crowd-funded open-source software

To help us develop this template and software sustainably under the MIT license, we ask all individuals and businesses that use it to help support its ongoing maintenance and development via sponsorship.

As a token of appreciation for sponsoring, you can unlock these awesome rewards and extra features 🦄✨

Ecosystem

Hugo Academic CLI: Automatically import publications from BibTeX

Inspiration

Check out the latest demo of what you’ll get in less than 10 minutes, or view the showcase of personal, project, and business sites.

Features

Page builder - Create anything with widgets and elements
Edit any type of content - Blog posts, publications, talks, slides, projects, and more!
Create content in Markdown, Jupyter, or RStudio
Plugin System - Fully customizable color and font themes
Display Code and Math - Code highlighting and LaTeX math supported
Integrations - Google Analytics, Disqus commenting, Maps, Contact Forms, and more!
Beautiful Site - Simple and refreshing one page design
Industry-Leading SEO - Help get your website found on search engines and social media
Media Galleries - Display your images and videos with captions in a customizable gallery
Mobile Friendly - Look amazing on every screen with a mobile friendly version of your site
Multi-language - 34+ language packs including English, 中文, and Português
Multi-user - Each author gets their own profile page
Privacy Pack - Assists with GDPR
Stand Out - Bring your site to life with animation, parallax backgrounds, and scroll effects
One-Click Deployment - No servers. No databases. Only files.

Themes

Wowchemy and its templates come with automatic day (light) and night (dark) mode built-in. Alternatively, visitors can choose their preferred mode - click the moon icon in the top right of the Demo to see it in action! Day/night mode can also be disabled by the site admin in params.toml.

Choose a stunning theme and font for your site. Themes are fully customizable.

License

Released under the MIT license.

The shape and simplicity biases of adversarially robust imagenet-trained cnns

Sat, 01 Aug 2020 00:00:00 +0000

The shape and simplicity biases of adversarially robust imagenet-trained cnns

Abstract

Increasingly more similarities between human vision and convolutional neural networks (CNNs) have been revealed in the past few years. Yet, vanilla CNNs often fall short in generalizing to adversarial or out-of-distribution (OOD) examples which humans demonstrate superior performance. Adversarial training is a leading learning algorithm for improving the robustness of CNNs on adversarial and OOD data; however, little is known about the properties, specifically the shape bias and internal features learned inside adversarially-robust CNNs. In this paper, we perform a thorough, systematic study to understand the shape bias and some internal mechanisms that enable the generalizability of AlexNet, GoogLeNet, and ResNet-50 models trained via adversarial training. We find that while standard ImageNet classifiers have a strong texture bias, their R counterparts rely heavily on shapes. Remarkably, adversarial training induces three simplicity biases into hidden neurons in the process of “robustifying” CNNs. That is, each convolutional neuron in R networks often changes to detecting (1) pixel-wise smoother patterns, i.e., a mechanism that blocks high-frequency noise from passing through the network; (2) more lower-level features i.e. textures and colors (instead of objects);and (3) fewer types of inputs. Our findings reveal the interesting mechanisms that made networks more adversarially robust and also explain some recent findings e.g., why R networks benefit from a much larger capacity (Xie et al. 2020) and can act as a strong image prior in image synthesis (Santurkar et al. 2019).

Layer-Wise Entropy Analysis and Visualization of Neurons Activation

Thu, 27 Feb 2020 00:00:00 +0000

Layer-Wise Entropy Analysis and Visualization of Neurons Activation

Abstract

Understanding the inner working mechanism of deep neural networks (DNNs) is essential and important for researchers to design and improve the performance of DNNs. In this work, the entropy analysis is leveraged to study the neurons activation behavior of the fully connected layers of DNNs. The entropy of the activation patterns of each layer can provide an efficient performance metric for the evaluation of the network model accuracy. The study is conducted based on a well trained network model. The activation patterns of shallow and deep layers of the fully connected layers are analyzed by inputting the images of a single class. It is found that for the well trained deep neural networks model, the entropy of the neuron activation pattern is monotonically reduced with the depth of the layers. That is, the neuron activation patterns become more and more stable with the depth of the fully connected layers. The entropy pattern of the fully connected layers can also provide guidelines as to how many fully connected layers are needed to guarantee the accuracy of the model. The study in this work provides a new perspective on the analysis of DNN, which shows some interesting results.

Layer-Wise Entropy Analysis and Visualization of Neurons Activation

Wed, 01 Jan 2020 00:00:00 +0000

The shape and simplicity biases of adversarially robust imagenet-trained cnns

Wed, 01 Jan 2020 00:00:00 +0000

Academic

The Annotated Transformer

Organize your notebooks

Import the notebooks into your site

PEEB, Part-based Bird Classifier with an Explainable and Editable Language Bottleneck

PEEB: Part-based Bird Classifier with an Explainable and Editable Language Bottleneck

Abstract

PEEB: Part-based Image Classifiers with an Explainable and Editable Language Bottleneck

Blog with Jupyter Notebooks!

Organize your notebooks

Import the notebooks into your site

gScoreCAM, What objects is CLIP looking at?

gScoreCAM: What is CLIP looking at?

Abstract

How explainable are adversarially-robust cnns?

How explainable are adversarially-robust cnns?

Abstract

gScoreCAM: What objects is CLIP looking at?

How explainable are adversarially-robust cnns?

Welcome to Hugo Blox Builder, the website builder for Hugo

Overview

Get Started

Crowd-funded open-source software

❤️ Click here to become a sponsor and help support Wowchemy’s future ❤️

Ecosystem

Inspiration

Features

Themes

License

The shape and simplicity biases of adversarially robust imagenet-trained cnns

The shape and simplicity biases of adversarially robust imagenet-trained cnns

Abstract

Layer-Wise Entropy Analysis and Visualization of Neurons Activation

Layer-Wise Entropy Analysis and Visualization of Neurons Activation

Abstract

Layer-Wise Entropy Analysis and Visualization of Neurons Activation

The shape and simplicity biases of adversarially robust imagenet-trained cnns