メインコンテンツまでスキップ

How to setup SFTP in EC2 allowing file upload

· 約2分
Mikyan
白い柴犬

SFTP use SSH protocal, allowing user to transfer files safely. It is a subsystem of SSH, so it runs in port 22.

The name contains FTP, however it is not implements FTP protocal

  • It achieves the same function with FTP
  • It is widely supported by FTP clients

When you want to transfer files with server, it might be a good choice.

How to setup a sftp user allowing it to upload files into specific folder

The following scripts helps setup a SFTP user in

Save it to .sh file, execute the following commands

chmod +x setup_sftp_user.sh
sudo ./setup_sftp_user.sh vendor

It will prompt you to set the password and configure the rest automatically.

#!/bin/bash

# Check if running as root
if [ "$EUID" -ne 0 ]; then
echo "Please run as root or use sudo"
exit 1
fi

# Check if username is provided
if [ -z "$1" ]; then
echo "Usage: $0 <username>"
exit 1
fi

USER=$1

# Create user
useradd -m $USER
passwd $USER

# Setup SFTP directories
mkdir -p /var/sftp/$USER/uploads
chown root:root /var/sftp/$USER
chmod 755 /var/sftp/$USER

chown $USER:$USER /var/sftp/$USER/uploads
chmod 755 /var/sftp/$USER/uploads

# Backup sshd_config
cp /etc/ssh/sshd_config /etc/ssh/sshd_config.bak

# Add SSH config for user
cat <<EOL >> /etc/ssh/sshd_config

Match User $USER
ChrootDirectory /var/sftp/$USER
ForceCommand internal-sftp
AllowTcpForwarding no
X11Forwarding no
EOL

# Restart SSH daemon
systemctl restart sshd

# Confirm success
echo "SFTP user '$USER' has been set up successfully."

Best Practice

· 約1分
Mikyan
白い柴犬

To simplify & unify the system logic, the following convention are considered as best practice:

  • Store everything in UTC
  • Use UTC everywhere in backend
  • Use UTC, with ISO-8601 format in API layer
  • Frontend: Convert to user's local time

How to do one thing well

· 約1分
Mikyan
白い柴犬

There are many situations, that we work on one thing that not familiar with. Or there is no known best practices. But luckily we can still have framework/principles to tackle them scientifically.

To do problems solving there are 4 basic elements:

  • Solving the real problem
  • Build a Causal Model for knowledge
  • Believe in the principle/best practice
  • Get feedback and iterate the knowledge

Science

Science is a systematic enterprise that builds and organizes knowledge in the form of testable explanations and predictions about the universe.

Theory Facts

7つの習慣ー主体的である

· 約1分
Mikyan
白い柴犬

7つの習慣の一番目の習慣は主体的であるです。

毎日起こることと反応の間で選択の自由を意識し、常に

我々の人生では毎日起こることがあります、その起こることに対して、私たちの反応があります。 その起こることと反応の間で、我々が選択する事由がある。 その選択する自由を意識し、

Python Pydantic

· 約3分
Mikyan
白い柴犬

Pydantic can extend standard Python classes to provide robust data handling features. BaseModel is the fundamental class in Pydantic. By inheriting from BaseModel, Python classes become Pydantic models, gaining capabilities for:

  • Data Validation: Automatically check the types and values of class attributes against your defined type hints. It raises a ValidationError with clear, informative messages if incoming data doesn't confirm.
  • Data Coercion: Pydantic can intelligently convert input data to the expected type where appropriate.
  • Instantiation: Creates instances of your model by passing keyword arguments or a dictionary to the constructor

Details

Inheriting from BaseModel, Python classes become Pydantic models

You can use Field function, or annotation to add more specific constraints and metadata to your fields.



from typing import Optional
from pydantic import BaseModel, Field, EmailStr

class User(BaseModel):
name: str
age: int
email: str

# Valid data
user = User(name="Alice", age=30, email="[email protected]")
print(user)

# Invalid data will raise a ValidationError
try:
User(name="Bob", age="twenty", email="bob@invalid")
except Exception as e:
print(e)


class Product(BaseModel):
id: int = Field(..., gt=0, description="Unique product identifier")
name: str = Field(..., min_length=2, max_length=100)
price: float = Field(..., gt=0.0)
description: Optional[str] = None # Optional field
seller_email: EmailStr # Pydantic's built-in email validation

product = Product(id=1, name="Laptop", price=1200.50, seller_email="[email protected]")
print(product)

Create Pydantic Model

Directly use Constructor with unpacked dictionary, or model_validate do validate and convert dict to model.

model_validate_json do validate and convert JSON string to a model.

user_data = {
"name": "Alice",
"age": 30,
"email": "[email protected]"
}
user_model = User(**user_data)

user_model = User.model_validate(user_data)


class Movie(BaseModel):
title: str
year: int
director: str
genres: list[str]

# Your JSON string data
json_string = '''
{
"title": "Inception",
"year": 2010,
"director": "Christopher Nolan",
"genres": ["Sci-Fi", "Action", "Thriller"]
}
'''
movie_model = Movie.model_validate_json(json_string)

Validate dictionary and JSON string: model_validate(), model_validate_json()

model_validate: validate a Python dictionary model_validate_json: validate a JSON string

from pydantic import BaseModel
import json

class Item(BaseModel):
name: str
quantity: int

data_dict = {"name": "Apple", "quantity": 5}
item1 = Item.model_validate(data_dict)
print(item1)

json_data = '{"name": "Banana", "quantity": 10}'
item2 = Item.model_validate_json(json_data)
print(item2)

Serialization: model_dump(), model_dump_json().

model_dump: to Python dictionary model_dump_json: to JSON

from pydantic import BaseModel

class City(BaseModel):
name: str
population: int

tokyo = City(name="Tokyo", population=14000000)
print(tokyo.model_dump())
print(tokyo.model_dump_json(indent=2)) # Pretty print JSON

Custom Validators: @field_validator, @model_validator

from datetime import date
from pydantic import BaseModel, ValidationError, field_validator, model_validator

class Event(BaseModel):
name: str
start_date: date
end_date: date

@field_validator('name')
@classmethod
def check_name_is_not_empty(cls, v):
if not v.strip():
raise ValueError('Event name cannot be empty')
return v

@model_validator(mode='after') # 'after' means after field validation
def check_dates_order(self):
if self.start_date > self.end_date:
raise ValueError('Start date must be before end date')
return self

try:
event1 = Event(name="Conference", start_date="2025-07-20", end_date="2025-07-22")
print(event1)
except ValidationError as e:
print(e)

try:
Event(name="Bad Event", start_date="2025-07-25", end_date="2025-07-23")
except ValidationError as e:
print(e)

Nested Models

from pydantic import BaseModel
from typing import List

class Address(BaseModel):
street: str
city: str
zip_code: str

class Customer(BaseModel):
customer_id: int
name: str
shipping_addresses: List[Address]

customer_data = {
"customer_id": 123,
"name": "Jane Doe",
"shipping_addresses": [
{"street": "123 Main St", "city": "Anytown", "zip_code": "12345"},
{"street": "456 Oak Ave", "city": "Otherville", "zip_code": "67890"}
]
}

customer = Customer.model_validate(customer_data)
print(customer)

JSON Schema Generation

from pydantic import BaseModel

class Task(BaseModel):
id: int
title: str
completed: bool = False

print(Task.model_json_schema(indent=2))

References

document on how to use it.

Python Type Hint

· 約2分
Mikyan
白い柴犬

Since Python 3.5, Python introduced Type hint. And it become more and more powerful.

With it You can set the type for your variable for readibility.

Type hints are hints, not enforcements. Python still runs the code even if types don't match.

Usage

# Primitives
name: str = "Tom"
age: int = 30
salary: float = 500.5
is_active: bool = True

# Collections
numbers: list = [1,2,3]
scores: tuple = (90, 85, 88)
unique: set = {1, 2, 3}
data: dict = {"key": "value"}


# Specific Collection Types

from typing import List, Dict, Tuple, Set

names: List[str] = ["Alice", "Bob", "Charlie"]
user: Dict[str, str] = {
"name": "John",
"email": "[email protected]"
}
person: Tuple[str, int, bool] = ("Alice", 30, True)
unique_ids: Set[int] = {1, 2, 3, 4, 5}

# after python 3.9 the following are also work
names: list[str] = ["Alice", "Bob", "Charlie"]
user: dict[str, str] = {
"name": "John",
"email": "[email protected]"
}person: tuple[str, int, bool] = ("Alice", 30, True)
unique_ids: set[int] = {1, 2, 3, 4, 5}

# Optional

from typing import Optional

# can be string or None
middle_name: Optional[str] = None

# Union
from typing import Union
number: Union[int, float] = 10
number = 10.5


# Literal for exact values
from typing import Literal
Status = Literal["pending", "approved", "rejected"]

def process_order(status: Status) -> None:
pass

# TypedDict
from typing import TypedDict
# TypedDict for dictionary structures
class UserDict(TypedDict):
name: str
age: int
email: str


# Class
user: User = get_user(123)

# method
def calculate_bmi(weight: float, height: float) -> float:
return weight / (height ** 2)

# Self
from typing import Self

class User:
def copy(self) -> Self: # Returns same class type
return User()

Java Development Setup, SDKMAN, Gradle, VS Code

· 約2分
Mikyan
白い柴犬

SDKMAN! is a popular Java version manager for developers. It allows you to easily install, switch between, and manage multiple Java versions and development tools.

Install SDKMAN!

# install
curl -s "https://get.sdkman.io" | bash
# init
source "$HOME/.sdkman/bin/sdkman-init.sh"

Install Java

# list available Java versions
sdk list java

# Install Java 21
sdk install Java 21.0.1-tem

# Set as default
sdk default java 21.0.1-tem

# Verify installation
java -version

Fix JAVA_HOME Environment Variable

SDKMAN! sometimes doesn't set JAVA_HOME automatically. Add this to your shell configuration:

# For zsh (default on newer macOS)
echo 'export JAVA_HOME="$SDKMAN_DIR/candidates/java/current"' >> ~/.zshrc
echo 'export PATH="$JAVA_HOME/bin:$PATH"' >> ~/.zshrc
source ~/.zshrc

# Verify JAVA_HOME is set
echo $JAVA_HOME

Install Gradle

sdk install gradle

gradle --verion

Install Spring boot

sdk install springboot

Start coding Java with VSCode

Install the Extension Pack for Java which includes 7 essential extensions:

  • Language Support for Java
  • Debugger for Java
  • Test Runner for Java
  • Maven for Java
  • Project Manager for Java
  • Visual Studio IntelliCode
  • Gradle for Java

Update user settings: Command + Shift + P > Open user settings (JSON) with the following settings:

{
// Java Runtime Configuration
"java.jdt.ls.java.home": "${env:JAVA_HOME}",
"java.configuration.runtimes": [
{
"name": "JavaSE-21",
"path": "${env:JAVA_HOME}",
"default": true
}
],
// Java Language Server Settings
"java.configuration.detectJdksAtStart": true,
"java.configuration.updateBuildConfiguration": "automatic",
"java.server.launchMode": "Standard",
"java.autobuild.enabled": true,
"java.maxConcurrentBuilds": 1,

// Code Completion and Analysis
"java.completion.enabled": true,
"java.completion.guessMethodArguments": true,
"java.completion.favoriteStaticMembers": [
"org.junit.jupiter.api.Assertions.*",
"org.mockito.Mockito.*",
"org.springframework.test.web.servlet.request.MockMvcRequestBuilders.*",
"org.springframework.test.web.servlet.result.MockMvcResultMatchers.*"
],
"java.compile.nullAnalysis.mode": "automatic",
"java.sources.organizeImports.staticStarThreshold": 5,
"java.sources.organizeImports.starThreshold": 5,

// Code Formatting
"java.format.enabled": true,
"java.format.settings.url": "https://raw.githubusercontent.com/google/styleguide/gh-pages/eclipse-java-google-style.xml",
"java.format.settings.profile": "GoogleStyle",
"java.format.comments.enabled": true,

// Gradle Settings
"java.import.gradle.enabled": true,
"java.import.gradle.wrapper.enabled": true,
"java.import.gradle.offline.enabled": false,
"java.import.gradle.arguments": "--refresh-dependencies",
"gradle.javaDebug.cleanOutput": true,
"gradle.debug": false,

// Testing
"java.test.defaultConfig": "",
"java.test.config": {},

// Debugging
"java.debug.logLevel": "warn",
"java.debug.settings.enableRunDebugCodeLens": true,
"java.debug.settings.showHex": false,
"java.debug.settings.showStaticVariables": false,
"java.debug.settings.showQualifiedNames": false,
"java.debug.settings.maxStringLength": 0,
"java.debug.settings.exceptionBreakpoint.skipClasses": [
"$JDK",
"junit.*",
"org.junit.*",
"org.springframework.*"
],

// Code Lens
"java.references.includeAccessors": true,
"java.references.includeDecompiledSources": true,
"java.implementationsCodeLens.enabled": true,
"java.referencesCodeLens.enabled": true,

// Error Reporting and Telemetry
"java.errors.incompleteClasspath.severity": "warning",
"java.telemetry.enabled": false,

// Performance
"java.jdt.ls.vmargs": "-XX:+UseParallelGC -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -Dsun.zip.disableMemoryMapping=true -Xmx2G -Xms100m",

// Code Completion Filtering (keep packages you don't commonly use out of suggestions)
"java.completion.filteredTypes": [
"java.awt.*",
"com.sun.*",
"sun.*",
"jdk.*",
"org.graalvm.*",
"io.micrometer.shaded.*",
"javax.swing.*"
],

// Spring Boot specific (useful for microservices)
"spring-boot.ls.checkjvm": false,
"spring.initializr.defaultLanguage": "Java",
"spring.initializr.defaultJavaVersion": "21",
"spring.initializr.defaultPackaging": "Jar",
}

Java Sealed

· 約2分
Mikyan
白い柴犬

After Java 8, the interface of java have become more and more powerful.

  • (Java 8) Default methods Provide optional functionality to all implementations
  • (Java 8) Static methods Utility methods that belong to the interface contract itself, CANNOT be overridden. utility functions that don't need instance context
  • (Java 9) Private methods: keeping internal helper logic hidden from implementers
  • Constants: Shared Contract Values public static final cannot be overriden
  • (Java 17) Sealed: Sealed means you can control exactly which classes can implement your interface (or extend your class). closed hierarchy, This enables better pattern matching, safer APIs, and clearer domain modeling!
  • (Java 8) @FunctionalInterface indicate this interface supports lambda.

Example:

public interface PaymentProcessor {
// Abstract method - must be implemented
void processPayment(double amount);

// Default method - optional to override
default void logTransaction(double amount) {
System.out.println("Processing payment of: $" + amount);
// Common logging logic here
}

// Another default method
default boolean validateAmount(double amount) {
return amount > 0 && amount < 10000;
}
}
public abstract class AbstractPaymentProcessor {
private int defaultAmount = 500;
// Abstract method - must be implemented
void processPayment(double amount);

// Default method - optional to override
void logTransaction(double amount) {
System.out.println("Processing payment of: $" + amount);
// Common logging logic here
}

// Another default method
boolean validateAmount(double amount) {
return amount > 0 && amount < 10000;
}
}