Python

Use Decorator in Python

Let’s grasp the concept of decorator in Python PEP 318 with short snippets. Step 1: Function as a parameter def f1(): print("Called f1.") def f2(f): f() f2(f1) # Called f1 f2 takes a function (object) as a parameter. Step 2: Wapping function def f1(fun): def wrap(): print("Start wrap") fun() print("End wrap") return wrap def f(): print("In function f") f1(f)() ### python test.py #Start wrap #In function f #End wrap Step 3: Use decorator def f1(fun): def wrap(): print("Start wrap") fun() print("End wrap") return wrap @f1 def f(): print("In function f") f() ### python test.

GraphQL tutorial with Python (`graphene`)

Original source I followed the linke below. https://hatemtayeb2.medium.com/hello-graphql-a-practical-guide-a2f7f9f70ab4 Install library pip -U install graphene Use it scheme.py import graphene import json class Query(graphene.ObjectType): hello = graphene.String() def resolve_hello(self, info): return "world" schema = graphene.Schema(query=Query) result = schema.execute( ''' { hello } ''' ) data = dict(result.data.items()) print(json.dumps(data,indent=2)) Try the code. $ python schema.py { "hello": "world" } GraphQL concepts Every GraphQL implementation needs a Query class that contains some fields, every field (not always) must have a resolver function that return data related to the field.

Pandas - cheetsheet

Basic information about dataframe df.info() #basic information about dataframe len(df.index) #rethrn the number of rows (data) df.count() #return the number of values which are non-NaN on each column df.head() df.tail() Count the data in a column In this example, the column is “Product”. df["Product"].value_counts() unique values to series. df["Product"].unique() # the type numpy.ndarray check distrivution in graph # Check the data distribution # The column is Score ax = df["Score"]value_counts().plot(kind='bar') fig = ax.

Py2PDF - read text from PDF file

Concept There are manuy useful PDF documents in the internet. These are very useful when we get data for training. PyPDF2 is a library for manipulating PDF files via Python. PyPDF2 Official Documentation Install You can install PyPDF2 via pip. pip install PyPDF2 How to use - read a PDF file PdfFileReader Class - Official PyPDF2 document We should open a file with mode rb. Read the file with PyPDF2.

SpaCy

What is spaCy In README of the GitHub project, there is a discription what is spaCy. spaCy: Industrial-strength NLP spaCy is a library for advanced Natural Language Processing in Python and Cython. It’s built on the very latest research, and was designed from day one to be used in real products. spaCy comes with pretrained statistical models and word vectors, and currently supports tokenization for 50+ languages. It features state-of-the-art speed, convolutional neural network models for tagging, parsing and named entity recognition and easy deep learning integration.

String manipulation with Python (for NLP)

f-strings Concept The official name of f-strings is “formatted string literal.” f-string is a “modern” way to put values of variables into the strings (so fat in Feb. 2020.) Before fstring appears, we were using format method. For me, fstring is very intuitive than format method. Formatted string literals - Python official document format - Python official document Simple example Both print lines in following code print the string “It’s me, Mario”.

NLP tools in Python

Libraries I need a few libraries for NLP and each of them are very powerful. I downloaded all of these libraries via pip, like pip install -U {package}. In the last section, I summaraized the libraries and I can install them at once later. spaCy: Open source NLP library. NLTK: Natural Language ToolKit. It is older than spaCy (spaCy 2015~, NLTK 2001~). gensim: NLP tools. I installed it for Doc2Vec.

pyenv and pyenv-virtualenv - Intro

Installing stacks You may be confused at first the differences between, pyenv virtualenv pyenv-virtualenv Especially, virtualenv sounds like Linux virtual environment, but it isn’t at all. Here is the good answer about that. Conclusion: pyenv-virtualenv is the best choice. pyenv-virtualenv official: https://github.com/pyenv/pyenv-virtualenv Install pyenv Install pyenv in macOS (or other Linux environments). # Download source under ~/.pyenv git clone https://github.com/yyuu/pyenv.git ~/.pyenv # Set PATH and another variable echo -e '\n export PYENV_ROOT=$HOME/.

Jupyter Notebook

Environment macOS: 10.15.3 Python: pyenv installed Install pip install -U pip pip install jupyter How to Run jupyter-notebook run the Jupyter. % jupyter-notebook [I 19:21:07.376 NotebookApp] Serving notebooks from local directory: /Users/atlex/nlp [I 19:21:07.376 NotebookApp] The Jupyter Notebook is running at: [I 19:21:07.376 NotebookApp] http://localhost:8888/?token=8637cf3f5b526b4ac33e93964844ecf7626ccd7be92616a2 [I 19:21:07.376 NotebookApp] or http://127.0.0.1:8888/?token=8637cf3f5b526b4ac33e93964844ecf7626ccd7be92616a2 [I 19:21:07.376 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation). [C 19:21:07.