minder_utils and lite

How to use this website

The documentation for each of the packages can be found in the toolbar on the left. The toolbar also holds some information about some of the IDs.

To return to this page at any point, click on the title and image in the top left corner.

The installation instructions and information about which package to install can be found below:

Which version is right for me?

Here, we have two versions of minder-utils. You only need one version!

  1. The first, is minder-utils-lite. This package just allows you to download the raw data from The Research Portal. This includes initial downloads and refreshes of the data on your machine. This version requires considerably less dependencies and is for those users that don't need extra models, pre-processing and visualisation packages.
  2. The second, is minder-utils and is the full version of the above. This package includes more dependencies because it is also packaged up with some machine learning models and visualisation functions. It also applied pre-processing to some of the data and downloads the raw data as well as saves .npy files. It also allows you to download the raw data from The Research Portal.

The installation instructions are as follows:

Installation instructions

minder-utils-lite

To install minder-utils-lite, simply run the following command:

pip install -e git+https://github.com/minder-utils/minder_utils_lite.git#egg=minder_utils

This will save a folder containing the package in your current working directory, with the name src. Please do not delete this!

The package will be installed under the name minder_utils. To import it, please run import minder_utils

minder-utils

If you are on Windows 10 and use the conda environment manager, then you may want to use the environment.yml file that is supplied within the package, located here: minder_utils package. Otherwise, please follow the instructions below.

If attempting to install this package on MacOS with M1 silicon then please first follow the instructions here: Installing Tensorflow on MacOS (Please ask Alex if you need help because this is quite unfriendly). Then you may move to step 2. If you are installing on Windows, move to step 1.

  1. Install tensorflow 2 using the command here: Tensorflow Installation Guide

    • The package has been tested with tensorflow 2.6.0 on windows and 2.4.0 on MacOS.

  2. Install pytorch, using the command here: Pyorch Installation Guide

    • The package has been tested with pytorch 1.9.1 with cuda 10.2 on windows and 1.9.1 on MacOS.

  3. Install scikit-learn using the command here: Scikit-Learn Installation Guide

    • The package has been tested with scikit-learn 1.0 on windows and 1.0 on MacOS.

  4. Install the minder_utils package using the following command:

    pip install -e git+https://github.com/ImperialCollegeLondon/minder_utils.git#egg=minder_utils

Then, you should be up and running! To import the package, please use import minder_utils.

If you would like to use the TIHM data in addition to the minder data, please download tihm.zip from the RDS, as well as validated_date.py. validated_date.py must be placed in a folder, with the path to this folder given to the function minder_utils.settings.set_data_dir. The TIHM data should be extracted from the zip file and the file path to the folder saved using minder_utils.settings.set_tihm_dir.

If you have any issues, please consult the troubleshooting below. If your answer can not be solved here, then please contact Alex or create an issue on our github for the corresponding package:

Troubleshooting instructions

There are many issues that arise from incompatibilities with Apple's M1 silicon.

  • If you are a MacOS M1 user and ran conda install jupyterlab and it won't work when you run jupyter lab. You also might need to run: conda install nbclassic==0.2.8 to reinstall part of the Jupyter package that is broken.
  • If scikit-learn keeps erroring, uninstall it and then do the following:
    • conda install scikit-learn
    • conda install scipy
  • After installing the packages on Windows 10, I had to install six==1.15.0, typing-extensions==3.7.4 and scipy, because tensorflow was erroring.