Python – Plot all predictions of the model multi-label classification

I want to show my input and output of the model I want to train:

Input data form:

processed_data.shape
(100, 64, 256, 2)

How it looks:

processed_data
array((((( 1.93965047e+04,  8.49532852e-01),
         ( 1.93965047e+04,  8.49463479e-01),

Form of the output data:

output.shape
(100, 6)

The output is basically the probability of each label

output = model.predict(processed_data)

output
array(((0.53827614, 0.64929205, 0.48180097, 0.50065327, 0.43016508,
        0.50453395)

I want to somehow represent the predicted class probabilities for each instance in processed data (since this is the problem of multi-label classification), but I have difficulty doing this.
So how I can plot the processed data but not sure how to represent the probabilities for each instance of the input. I want to be able to identify all 6 possible classes on every issue. I'm a little lost …
Any suggestions?

So far I only draw inputs:
shape = output.shape (0)

for i in range(it):
    fig,axs = plt.subplots(5,2,figsize=(10,10))

    if isinstance(data,list): 
        inp = data(i)
        outp = output(i)
    else: 
        inp = data
        outp = output

    for j in range(5):
        r = randint(0,shape)
        axs(j,0).imshow(inp(r,...,0)); 
        axs(j,0).title.set_text('Input {}'.format(r))

Python – representation of a continuous wordbag model with Numpy

I'm trying to build a network that models a Continuous Bag of Words model in NumPy, and I'm trying to figure out where I made a mistake here. My network doesn't seem to be learning at all and predicts that every sentence contains the same word. Of course there is something that I did wrong, but I cannot say exactly where.

So my approach is as follows:

  • Tokenize the body in two vectors (contexts and targets). Each set of elements in the context window in the contexts The vector is assigned to an entry in the field targets. So these are mine X_train and Y_train. For example with a context size of 2the words ("Hello", "How", "You", "Do") can be assigned to the target word "are" because it is the middle word of the body ("Hello", "How", "are" "," you do "). The middle word is the goal and the words +-CONTEXT_SIZE belong contexts.

  • Get the indexing of the tokens from the corpus and pass them to the embedded embedding layer (vocab_size, embedding_size). The bedding layer is a lookup table from which to map vocab_size to embedding_size Dimensions, so I'm going to look at the embedding vector that represents them average contextual words, i.e. word_vec. So this vector has a shape (1, embedding_size) (A column vector).

  • Now I'm coming past word_vec (average embedded word vector) in a hidden layer (with shape) embedding_size, vocab_size). That has one tanh Activation function with a preload. So I have to normalize my vector after multiplying it by W. The output vector of this level has the form (1, vocab_size)Basically, I can simply pass this on to my Softmax function.

  • After getting the word index with softmax, I find the maximum probability and predict the output. Should be okay, right?

My gradients seem to tend to explode, but the clipping doesn't seem to fix anything. I will post the full code below. I would be grateful if someone could point me to the right steps. Was it an initialization error or did I do something wrong elsewhere?

import numpy as np
import re
from nltk.corpus import brown
import matplotlib.pyplot as plt

class CBOW:
    def __init__(self, contexts, targets, vocab_size, embedding_size, learning_rate, context_size=3):
        # Initialize Stuff
        self.embedding_matrix = np.random.randn(vocab_size, embedding_size) * 0.01
        self.num_classes = vocab_size
        self.W = np.random.randn(embedding_size, self.num_classes) * 0.01
        self.bias = np.zeros((1, self.W.shape(1)))
        self.word_vec = None
        self.gradients = dict()
        self.cache_contexts = None # For backprop
        self.cache_Z = None # For backprop
        self.learning_rate = learning_rate
        self.contexts = contexts
        self.targets = targets
        self.context_width = 2*context_size


    def Embedding(self, contexts, train):
        # Op.shape = 1 * embedding_size (Column Vector)
        # Take the average vector from the context window
        word_vec = np.mean(self.embedding_matrix(contexts, :), axis=1)

        #assert word_vec.shape == (self.context_width, self.embedding_matrix.shape(1))
        assert word_vec.shape == (1, self.embedding_matrix.shape(1))
        self.word_vec = word_vec


    def tanh(self, inp):
        return np.tanh(inp)


    def tanh_delta(self, inp):
        return (1 - inp*inp)


    def Dense(self, word_vec):
        # Pass the word_vec thru the Dense Layer
        op = np.dot(word_vec, self.W) + self.bias
        # Normalize before passing to tanh
        op = op / (np.sum(1e-2 + np.max(op), keepdims=True))
        op = self.tanh(op)
        assert op.shape == (1, self.W.shape(1))
        return op


    def softmax(self, inp):
        assert inp.shape == (1, self.num_classes)
        max_val = np.max(inp, axis=1)
        softmax_out = np.exp(inp - max_val) / (np.sum(np.exp(inp - max_val), keepdims=True) + 1e-3)
        return softmax_out


    def forward(self, contexts, train):
        self.Embedding(contexts, train)
        Z = self.Dense(self.word_vec)
        self.cache_Z = Z
        softmax_out = softmax(Z)
        self.cache_contexts = contexts
        return softmax_out


    def cross_entropy(self, softmax_out, Y):
        target = np.zeros(softmax_out.shape)
        target(:, np.squeeze(Y)) = 1
        loss = -(target * np.log(softmax_out + 1e-3))
        return np.max(loss)


    def backward(self, Y, softmax_out):
        # Perform backprop
        # Z = tanh(O)
        # O = W*word_vec + bias
        # softmax_out = softmax(Z)
        # dL/d(Z) = (softmax_out - Y)

        Z = self.cache_Z
        target = np.zeros(softmax_out.shape)
        target(:, np.squeeze(Y)) = 1

        dL_dZ = softmax_out - target
        assert dL_dZ.shape == (1, self.num_classes)
        self.gradients('dL_dZ') = dL_dZ
        # dZ_dO = (1 - tanh^2(O)) = (1 - z*z)
        dZ_dO = self.tanh_delta(Z)
        assert dZ_dO.shape == (1, self.W.shape(1)) # (1, num_classes)

        # dL/dO = dL_dZ * dZ/dO
        dL_dO = dL_dZ * dZ_dO
        assert dL_dO.shape == (1, self.W.shape(1))
        self.gradients('dL_dO') = dL_dO

        # dL/d(W) = dL/d(O) * d(O)/d(W)
        # d(O)/dW = word_vec
        dO_dW = self.word_vec
        dL_dW = np.dot(dO_dW.T, dL_dO)
        assert dL_dW.shape == self.W.shape # (embedding_size, num_classes)
        self.gradients('dL_dW') = dL_dW
        # dL/d(word_vec) = dL/dO * dO/d(word_vec) = dL/dO * W
        dL_dword_vec = np.dot(dL_dO, self.W.T)
        assert dL_dword_vec.shape == self.word_vec.shape # (1, embedding_size)
        self.gradients('dL_dword_vec') = dL_dword_vec
        # dL/d(bias) = dL/dO * dO/d(bias) = dL/dO
        dL_dbias = dL_dO
        assert dL_dbias.shape == self.bias.shape
        self.gradients('dL_dbias') = dL_dbias

        # Clip all gradients
        for grad in self.gradients:
            self.gradients(grad) = np.clip(self.gradients(grad), -500, 500)
        #print(self.gradients)


    def update(self):
        contexts = self.cache_contexts
        dL_dword_vec = self.gradients('dL_dword_vec')
        self.embedding_matrix(contexts) -= self.learning_rate * dL_dword_vec
        self.W -= self.learning_rate * self.gradients('dL_dW')
        self.bias -= self.learning_rate * self.gradients('dL_dbias')


    def train(self, epochs):
        losses = ()

        X = self.contexts
        Y = self.targets

        vocab_size = self.num_classes

        context_width = Y.shape(1)

        for epoch in range(epochs):
            epoch_loss = 0
            factor = (2 * CONTEXT_SIZE)
            inds = list(range(0, context_width))
            np.random.shuffle(inds)
            print(f"Item #{inds(0)}")
            for i in inds:
                X_item = X(:, i*factor:(i+1)* factor)
                Y_item = Y(:, i:i+1)
                softmax_out = self.forward(X_item, train=True)
                self.backward(Y_item, softmax_out)
                self.update()
                loss = self.cross_entropy(softmax_out, Y_item)
                epoch_loss += np.squeeze(loss)
            losses.append(epoch_loss)
            if epoch:
                print(f"Loss after epoch #{epoch}: {epoch_loss}")
        plt.plot(np.arange(epochs), losses)
        plt.xlabel('# of epochs')
        plt.ylabel('cost')

Initialize and train the network with:

model = CBOW(contexts, targets, vocab_size, embedding_size=50, learning_rate=0.05)
model.train(100)

Here is the code for dealing with the body:

def tokenize(text):
    pattern = re.compile(r'(A-Za-z)+(w^')*|(w^')*(A-Za-z)+(w^')*')
    try:
        return pattern.findall(text.lower())
    except:
        return pattern.findall(text)

def generate_mapping(tokens):
    # Generates the word_to_idx and idx_to_word mappings
    word_to_idx, idx_to_word = {}, {}

    for idx, token in enumerate(set(tokens)):
        word_to_idx(token) = idx
        idx_to_word(idx) = token

    return word_to_idx, idx_to_word

def get_training_data(tokens, word_to_ix, CONTEXT_SIZE):
    X, Y = (), ()
    for i in range(CONTEXT_SIZE, len(tokens) - CONTEXT_SIZE):
        for j in range(i - CONTEXT_SIZE, i + CONTEXT_SIZE + 1):
            if j == i: continue
            Y.append(word_to_ix(tokens(j)))
        X.append(word_to_ix(tokens(i)))
    X, Y = np.array(X), np.array(Y)
    X, Y = X(np.newaxis, :), Y(np.newaxis, :)
    assert 2 * CONTEXT_SIZE * X.shape(1) == Y.shape(1)
    return X, Y

WINDOW_SIZE = 3
CONTEXT_SIZE = 3

# Taken from the PyTorch tutorial section on Word Embeddings
corpus = """We are about to study the idea of a computational process.
Computational processes are abstract beings that inhabit computers.
As they evolve, processes manipulate other abstract things called data.
The evolution of a process is directed by a pattern of rules
called a program. People create programs to direct processes. In effect,
we conjure the spirits of the computer with our spells."""

corpus_tokens = tokenize(corpus)
word_to_idx, idx_to_word = generate_mapping(corpus_tokens)
targets, contexts = get_training_data(corpus_tokens, word_to_idx, WINDOW_SIZE)
vocab_size = len(set(corpus_tokens))

Here is the code for testing on the same body by splitting it into tokens

def get_max_idx(inp):
    # Get counts and indices
    c = np.array(np.unique(np.argmax(probs, axis=1), return_counts=True))
    # Get the maximum count
    idx = np.argmax(c(1, :))
    return c(0, idx)

def get_index_of_max(inp):
    return np.argmax(inp, axis=1)

def get_max_prob_result(inp, ix_to_word):
    return ix_to_word(get_max_idx(inp))

def make_context_vector(context, word_to_idx):
    return np.array(((word_to_idx(word) for word in tokenize(context))))

def generate_model_probs(model, context_vector, train):
    softmax_out = model.forward(context_vector, train)
    return softmax_out

def test(model, corpus, word_to_idx, idx_to_word):
    corpus_tokens = tokenize(corpus)
    corpus_size = len(corpus_tokens)
    for i in range(0 + CONTEXT_SIZE, corpus_size - CONTEXT_SIZE):
        word = " ".join((corpus_tokens(j) for j in range(i - CONTEXT_SIZE, i + CONTEXT_SIZE + 1) if j != i))
        print("Word:" + word)
        context_vector = make_context_vector(word, word_to_idx)
        probs = generate_model_probs(model, context_vector, train=False)
        print(f"Prediction: {get_max_prob_result(probs, idx_to_word)}")
    print(probs)

Client-server model between smartphones via BLE?

I want to have a client-server model with smartphones in a range of 50 meters in one application. One of the clients is said to be the "host" and its only job is to receive data from and do something with all clients (not important). Everyone should be able to see all the other smartphones on which the app started on a list. In addition, each client should be able to see which of the clients in the list is the "host" to send data to.

I want the application to be decentralized so that it doesn't depend on a database or anything else, but only on the few people who use it within 50 meters. The advantage I get from this is that several groups in different locations can use the app.

I have problems with the technology on which I want to build this setup.

My thoughts on Bluetooth low energy:

  • easy to find, all people within reach
  • not dependent on wifi / internet
  • Problem: The clients cannot determine the host
  • Problem: There are many other Bluetooth devices that are not smartphones and should be ignored

Is there a better technique I could use? The disadvantage of WiFi is that everyone has to be there.

graph – Domain-specific language to describe the ensemble model

I am looking for a tool / library / widely used approach to describe the hierarchical model structure like the ensemble:
Example hierarchical model

It is absolutely simple how to do it with a simple ensemble like
simple ensemble

It can even be explained in yml format as follows:

code: "cats"
models:
  - code: "siamese"
  - code: "persian"
  - code: "sphynx"

Code that runs a simple ensemble model shows that exit should be done first and the parent model aggregates its results.

However, it is difficult to describe a hierarchical model in the same way because it runs in the opposite direction. Probably can someone suggest how to do it?

model – declaration warning when extending _afterLoad () in AbstractModel

I created a backend module to which I added the ability to set Storeview specific data today. That worked and went well. I'm now trying to load and save my storeview data, but it just doesn't work. I have taken the code from this answer and am using Magento 2.3.3.

Here is my model code in which I try to load and save the Storeview settings:

_init('CompanyModuleModelResourceModelModule');
    }

    /**
     * @param MagentoFrameworkModelAbstractModel $object
     * @return void
     */
    protected function saveStore($object)
    {
        $condition = $this->getConnection()->quoteInto('item_id = ?', $object->getId());
        $this->getConnection()->delete($this->getTable('company_module_store'), $condition);
        foreach ((array)$object->getData('store_id') as $store) {
            $storeArray = (
                'item_id' => $object->getId(),
                'store_id' => $store,
            );
            $this->getConnection()->insert(
                $this->getTable('company_module_store'),
                $storeArray
            );
        }
    }

    /**
     * @param MagentoFrameworkModelAbstractModel $object
     * @return $this
     */
    protected function _afterSave(AbstractModel $object)
    {
        if (!$object->getIsMassStatus()) {
            $this->saveStore($object);
        }

        return parent::_afterSave($object);
    }

    /**
     * @param MagentoFrameworkModelAbstractModel $object
     * @return MagentoFrameworkModelAbstractModel
     */
    public function loadStore(AbstractModel $object)
    {
        $select = $this->getConnection()->select()
            ->from($this->getTable('company_module_store'))
            ->where('item_id = ?', $object->getId());

        if ($data = $this->getConnection()->fetchAll($select)) {
            $array = ();
            foreach ($data as $row) {
                $array() = $row('store_id');
            }
            $object->setData('store_id', $array);
        }

        return $object;
    }

    /**
     * @param MagentoFrameworkModelAbstractModel $object
     * @return $this
     */
    protected function _afterLoad(AbstractModel $object)
    {
        $object = $this->_eventObject;
        if (!$object->getIsMassDelete()) {
            $this->loadStore($object);
        }

        return parent::_afterLoad($object);
    }
}

If I load this in the backend, I get:

Exception No. 0 (exception): Warning: The declaration of Company Module Model Module :: _ afterLoad (Magento Framework Model AbstractModel $ object) should be done with Magento Framework Model AbstractModel :: _ afterLoad ( ) be compatible.

I think the problem is that _afterload originally doesn't accept arguments? But how can I solve my problem? Really baffled here.

Java – Do you need a domain model in a 3-tier architecture service?

I am creating an API rest with Spring Boot and want to clarify a concept about my architecture to see if the community can help me.

Imagine that I have a table in my database Person. I assemble an architecture that is based on a three-layer architecture. The general scheme would be as follows:

  • On the one hand, I have that PersonDao.javaresponsible for accessing the database to retrieve the tuples from the database Person Table. For this purpose, a class called is called PersonEntity.javathat contains (as attributes) all columns of the table. PersonDao.java will return an object from the PersonModel.java Class. PersonModel.java is the class that represents it Person Business model.

  • On the other hand, I have that PersonService.javawho is responsible for executing the business logic of my application. This service is calling PersonaDao.java to access the information stored in the database. PersonService.java works with objects of PersonModel.java Class because this class represents the business objects of my application. PersonService.java will always return a PersonDto,java.

  • Finally, PersonController.java. This controller is the one that exposes the connection interface of the rest API. He always works with DTO and communicates with PersonService.java also via DTO.

PersonController <-> (PersonDto) <-> PersonService <-> (PersonModel) <-> PersonDao <-> (PersonEntity) <-> DB

The question is: is it necessary to use that PersonModel.java Class so that PersonService.java only works with objects of this class? Or would it be better for PersonService.java work directly with objects from the class PersonEntity.java? If I do so, it is the principle of individual responsibility that each level only works with objects of its scope.

If the answer is that PersonModel.java is necessary to maintain the principle of individual responsibility for each shift. Would change something if JpaRepository were used? When I ask this question, it is because I see when in many tutorials and examples JpaRepository the services work directly with the entities. In this case, shouldn't we create a class that represents the business object for the services?

Thank you in advance.

Python – My custom Django user model

I followed a video on YouTube to create a custom user model. This is my attempt:

# Django Imports
from django.contrib.auth.models import AbstractBaseUser, BaseUserManager
from django.db import models

# The Custom (UserManager) Class
class UserManager(BaseUserManager):
    # Defining A Function That Will Be Used To Create New Users
    def create_user(self, email, username, f_name, l_name, password = None):
        if not email:
            raise ValueError('Users Must Have An E-Mail Address')
        if not username:
            raise ValueError('Users Must Have A Username')

        # Creating An Object Of The Type (User)
        user                    = self.model(
            user_email          = self.normalize_email(email),
            user_name           = username,
            first_name          = f_name,
            last_name           = l_name
        )

        # Setting The Password For The User Object We Created Above
        user.set_password(password)

        # Saving The User To The Database
        user.save(using = self._db)

        # Returning The (user) Instance To Where The Function Was Called
        return user

    # Defining A Function That Will Be Used To Create Superusers (Admins)
    def create_superuser(self, email, username, f_name, l_name, password):
        # Creating An Instance Of Type (User)
        user                    = self.create_user(
            user_email          = self.normalize_email(email),
            user_name           = username,
            password            = password,
            first_name          = f_name,
            last_name           = l_name
        )

        # Setting The User We Created Above As An Administrator
        user.is_admin           = True
        user.is_staff           = True
        user.is_superuser       = True

        # Saving The User To The Database
        user.save(using = self._db)

        # Returning The (user) Instance To Where The Function Was Called
        return user

# The Custom (User) Class
class User(AbstractBaseUser):
    # The Data (Columns) For Each User
    user_email                  = models.EmailField(max_length = 120, unique = True)
    user_name                   = models.CharField(max_length = 40, unique = True)
    first_name                  = models.CharField(max_length = 60)
    last_name                   = models.CharField(max_length = 60)
    date_joined                 = models.DateTimeField(auto_now_add = True)
    last_login                  = models.DateTimeField(auto_now = True)
    is_admin                    = models.BooleanField(default = False)
    is_active                   = models.BooleanField(default = True)
    is_staff                    = models.BooleanField(default = False)
    is_superuser                = models.BooleanField(default = False)

    # The Data (Column) That Is Going To Be Used As A Login Identifier
    USERNAME_FIELD              = 'user_email'
    REQUIRED_FIELDS             = ('user_name', 'first_name', 'last_name')

    # Defining How The Output Of An Object Of The (User) Class Will Look Like
    def __str__(self):
        return f'{self.user_email} - ({self.user_name})'

    # Defining A Function That Will Check Whether A User Is Admin Or Not
    def has_perm(self, perm, obj = None):
        return self.is_admin

    # Defining A Function That Will Check Whether A User Is Admin In The Current App Or Not
    def has_module_perms(self, app_label):
        return True

I know that the way I comment on things is wrong. I will give up this habit because I don't have to write comments everywhere, but have to focus on the clarity of the code. Aside from my bad comment style, is this approach of creating custom users in Django correct?

P.S: I use PostgreSQL as my database

c # – DDD this (database) logic belongs to the model

I'm trying to figure out where to put the business logic and why and where the performance fits. I try to distance myself from the fat, logic-filled repositories that we often produce in my company.

So I'm trying to redesign this method in DDD style:

var doesCollide = _bookingRepository.HasCollisionWithOtherBookings(bookingEntity, newEnd);

What I think I want is the following:

var doesCollide = booking.CollidesWithOtherBookings(allBookings, newEnd);

The internal logic only does a classic date math. I still have to load allBookings from the database.

var allBookings = context.bookings.ToList();
var doesCollide = booking.CollidesWithOtherBookings(allBookings, newEnd);

But now I think ALL bookings? I can significantly improve performance by adding just a few simple conditions to my query.

var allBookings = context.bookings.Where(x => x.Start < newEnd && booking.Start < x.End && x.Id != booking.Id).ToList();
var doesCollide = booking.CollidesWithOtherBookings(allBookings, newEnd);

Big. But I literally just duplicated the exact logic that I tried to hide in CollidesWithOtherBookings. What does that mean anyway? Do I have to test this new query somehow?

I'm not sure what I'm testing here. The datetime logic is easy to mix up, so I should probably test it. When I test Booking.CollidesWithotherBookings, that's great, but in my actual application, I might have a bug because I can mess up the where clause.